The multi-parameter estimation of discrete distribution without closed-form solutions by the US algorithm

1.   Introduction

Discrete lifetime data—such as the number of appliance failures of a particular brand within a given time frame, the total number of machine operations prior to a failure, the number of bullets fired by a weapon before the first malfunction, and the anticipated lifespan of humans (in years)—are frequently handled in reliability lifetime studies. For more classic examples, see Szymkowiak and Iwinska ^[1]. Data scientists typically employ discrete models as analysis tools, such as the Poisson distribution, negative binomial distribution, and geometric distribution, in order to more correctly define, analyze, and model these data. But in many situations, these discrete distribution functions are not the best options. For instance, seasonal or periodic data cannot be handled by the Poisson distribution, while underdispersed data cannot be described by the negative binomial distribution. More suitable discrete lifetime distributions are required to explore many additional kinds of complex discrete lifespan data. Discretizing continuous random variables is a useful strategy that yields a discrete life model with characteristics that are comparable to the continuous model.

The essential concept of discretizing continuous random variables was first presented by Roy ^[2]. Specifically, let $Y$ be a continuous random variable with a survival function denoted by $S(y)$. Define the random variable $Z = [Y]$ as the maximum integer less than or equal to $Y$. The probability mass function (PMF) $P(Z = z)$ of $Z$ can be expressed as

Many researchers have introduced various new models for discrete life distributions by the approach. For instance, the discrete normal distribution was first introduced by Roy ^[3]. Using the generic method of discretizing a continuous distribution, Krishna and Pundir ^[4] introduced the discrete Burr and Pareto distributions. In addition, Bracquemond and Gaudoin ^[5] provided an extensive overview of discrete distributions, such as the Weibull distribution, that are employed in reliability to describe discrete lifetimes of nonrepairable systems. It is well-known that the Weibull distribution has become the most commonly used distribution for analyzing continuous life data due to its ability to fit various types of data and relatively simple structure (Johnson et al. ^[6]). At least three cases exist for the corresponding discrete Weibull distribution: (a) the Type Ⅰ discrete Weibull distribution, which maintains the form of the continuous survival function (SF), as introduced by Nakagawa and Osaki ^[7]; (b) the Type Ⅱ discrete Weibull distribution, as suggested by Stein and Dattero ^[8]; and (c) the three-parameter discrete Weibull distribution, as introduced by Padgett and Spurrier ^[9]. The most popular of them is the Type Ⅰ discrete Weibull distribution, whose features have been extensively researched by numerous academics. Englehardt and Li ^[10] employed the discrete Weibull distribution to analyze pathogen counts in treated water over time. Barbiero ^[11,12] compared several parameter estimation methods of this distribution, and solved the minimum Chi-square and least squares estimation. Vila et al. ^[13] studied in detail the basic theoretical properties of the Type Ⅰ discrete Weibull and analyzed the censored data. Yoo ^[14] extended the application of the discrete Weibull regression model to accommodate missing data. In addition, El-Morshedy et al. ^[15] conducted a detailed study on a new bivariate exponential discrete Weibull distribution.

The primary goal in this study is to enhance the current techniques for estimating the complex discrete probability distribution model. The probability distribution's score function typically lacks an explicit analytical solution, hence the Newton approach is usually used to estimate the numerical solution for parameter estimation. Nevertheless, the algorithm's low convergence and strong dependence on the initial value make it challenging to achieve the best estimation outcomes. Recently, Liu et al. ^[16] employed the majorize minimize (MM) algorithm to enhance the resolution of the maximum likelihood estimation for the simplex distribution. Li and Tian ^[17] introduced a novel root-finding method known as the upper-crossing/solution (US) algorithm. In contrast to conventional iterative algorithms (like Newton's algorithm), the US algorithm can lessen the influence of initial values and achieve a strong, stable convergence to the objective equation's real root at each iteration. The benefits of this technique have been illustrated through the use of a few classic models, such as the Weibull distribution, gamma distribution, zeta distribution, and generalized Poisson distribution. Cai ^[18] has improved the maximum likelihood estimation of generalized gamma distribution parameters by combining the US algorithm with the second-derivative lower-bound function (SeLF) algorithm.

The essence of the US algorithm is to identify a $U$-function $U(\theta|\theta^{(t)})$, which simplifies the solution of the complicated nonlinear equation $h(\theta) = 0$ to the solution of the equation $U(\theta|\theta^{(t)}) = 0$ with an explicit solution. Li and Tian ^[17] presented a variety of approaches to discover the $U$-function, among which the first-derivative lower bound (FLB) function method requires only by using the first derivative of the objective function $h(\theta)$, thereby diminishing algorithmic complexity. In previous research on the US algorithm, it was generally used to solve the roots of univariate nonlinear equations or the maximum likelihood estimation problem of a multi-parameter probability distribution with an explicit partial score function. Specifically, for a probability distribution with two parameters $(\alpha, \beta)$, while solving for maximum likelihood estimation, the estimator of the parameter $\alpha$ can be explicitly expressed by the other estimator of the parameter $\beta(\alpha)$. However, there is no further discussion provided in Li and Tian ^[17] about whether this approach can be applicable in more complex multi-parameter distributions, where an estimator of one parameter cannot be clearly represented by the other, and it is an issue deserving of more investigation.

The rest of the paper will proceed as follows. Section 2 provides a detailed introduction to the US algorithm and FLB function method. The application of the suggested approach to the Type Ⅰ discrete Weibull distribution's maximum likelihood parameter estimation is covered in Section 3. Numerical simulation experiments will be conducted in Section 4 in order to evaluate the performance of the employed methods and compare them with alternative estimation approaches. Section 5 will demonstrate the applicability of the US algorithm through the analysis of two real data sets. Conclusions and discussions will be provided in Section 6.

2.   The US algorithm

One of the most frequent issues in numerical computations is figuring out the zero point of a function or an equation's root. In classical statistics, the maximum likelihood estimate (MLE) of parameters and the calculation of maximum a posteriori probability in Bayesian statistics may typically be turned into the problem of solving the zero point of a nonlinear function $h (\theta)$. In summary, since $h(\theta)$ is a nonlinear function of a single variable $\theta$, we must identify the unique root $\theta^\star$ such that

The US algorithm is the most recent method for discovering roots. It has a similar procedure to the commonly used EM (expectation maximum) and MM (maximize minimize) algorithms^[17]. There are two primary steps in this process: the upper-crossing step (U-step) and the solution step (S-step). The two primary advantages of this algorithm are as follows:

(a) It converges strongly and stably to the root $\theta^\star$ of the Eq (2.1) with each iteration, that is, for an iterative points set sequence $\left\{\theta^{(t)} \right\}^{\infty}_{t = 0}$, there is

(b) The Newton algorithm's sensitivity to the initial value is decreased.

Two new symbols, $\overset{sgn(\alpha)}{\leq}$ and $\overset{sgn(\alpha)}{\geq}$, are introduced regarding the changing direction (CD) inequalities in order to simplify the explanation of the US algorithm. The specific definition is presented as follows: for two functions $f_1(x)$ and $f_2(x)$ on the same domain $\mathbb{Q}$,

and

2.1.   Definition of the U-function

It is typically challenging to locate the root $\theta^\star$ of the nonlinear equation $h(\theta) = 0$ directly. The US algorithm aims to create an alternative function $U(\theta|\theta^{(T)})$ to replace $h (\theta)$, transforming the challenge of solving complex nonlinear equations into solving the equation $U(\theta|\theta^{(T)})$ with explicit solutions. First, we assume that

If $h(\theta) < 0$ when $\theta < \theta^\star$, then both sides of the equation $h(\theta) = 0$ can be multiplied by -1, which can also obtain the same root $\theta^\star$ satisfying the assumption (2.2). Let $\theta^{(t)}$ represent the solution after the ($t$-1)-th iteration, and the function $U(\theta|\theta^{(t)})$ satisfying the following criteria is designated as the U-function of $h(\theta)$ at $\theta = \theta^{(t)}$:

According to the definition of the CD inequalities symbol, the above condition may be represented as

2.2.   The U-equation

As described above, the US algorithm is an iterative approach for solving nonlinear equations, with each iteration including a U-step and an S-step. The purpose of the U-step is to find a U-function that satisfies the condition (2.4), whereas the S-step involves solving the simplified U-equation: $U(\theta|\theta^{(t)}) = 0$ to obtain its root $\theta^{(t+1)}$,

In typical scenarios, $\theta^{(t+1)}$ can be explicitly expressed, even as a linear equation. Through the iterative execution of these two steps, $\left\{\theta^{(t)} \right\}^{\infty}_{t = 0}$ can gradually converge to the real root $\theta^\star$ of the U-equation.

2.3.   The first-derivative lower bound function method

There are numerous U-functions for a given objective function $h(\theta)$; as Eq (2.4) illustrates, distinct U-functions correlate to distinct US algorithms. We may express the U-function using the lower-order derivatives of the goal function $h(\theta)$. This can be accomplished by a variety of techniques, such as the first-derivative lower bound (FLB), second-derivative lower-upper bound (SLUB) constants method, and third-derivative lower bound (TLB) constant method ^[17]. These three methodologies enhance efficient solutions when the objective function is complex and the solution is not closed, each with a distinct convergence speed. In terms of maximizing the objective function, the US algorithm based on the FLB approach shares qualities with the EM algorithm and the MM algorithm, both of which exhibit linear convergence. The FLB function technique, which is dependent on the target function's first-order derivative, is mostly used in this article to generate the required U-function. First for parameter space $\Theta$, we suppose that there exists a certain first-derivative lower bound function $b(\theta)$ for the first derivative of $h(\theta)$, i.e.,

The U-function of $h(\theta)$ at $\theta = \theta^{(t)}$ can be formally defined as follows

In fact,

Let $\theta^\star$ be the unique root of the equation $h(\theta) = 0$, and then the corresponding US iteration is as follows:

where $g(\theta^{(t)}) = g(\theta^\star)+(\theta^{(t)}-\theta^\star)h^\prime(\theta^\star)+0.5(\theta^{(t)}-\theta^\star)^2h^{\prime\prime}(\hat{\theta})$ is the first-order Taylor expansion around $\theta^\star$, and $\theta^\star$ is a point between $\theta^{(t)}$ and $\theta^\star$.

Although Li and Tian ^[17] proposed the idea of the US algorithm, in practical applications, only the distribution of univariate and binary parameters were studied. In the case of binary parameter distribution, when discussing the solution of the scoring equation, only one parameter can be explicitly expressed with another parameter. However, when one parameter cannot be explicitly expressed by the other parameter for this more general and complex situation, whether the US algorithm can be effectively applied is not further discussed, which is the issue to be carried out in this article. For the parameters of interest, the new FLB functions are constructed in this article, then, starting with initial values, the iterative values are updated using the corresponding S-step until the convergence criteria are met.

3.   The US algorithm for the MLE of the Type Ⅰ discrete Weibull distribution

The discrete Weibull distribution's score function has a complex double exponential form, which makes it impossible to depict its solution and, thus, prevents its two parameters from being mutually expressed. For investigating the US algorithm's applicability in complicated models, we combine the US algorithm with the FLB method in this section to optimize the maximum likelihood estimation.

3.1.   The Type Ⅰ discrete Weibull distribution

Assuming a random variable following the Weibull distribution $W(\lambda, \beta)$, where $\lambda > 0$ and $\beta > 0$, the cumulative distribution function (CDF) of the Weibull distribution is defined as $H\left(t, \lambda, \beta\right) = 1-e^{-\lambda t^\beta}$, where $t > 0$. Define $\alpha = e^{-\lambda}$, and then $0 < \alpha < 1$. If the probability mass function (PMF) for a random variable $X$ can be represented as

then we say that $X$ follows the Type Ⅰ discrete Weibull distribution, denoted as $X\sim DW(\mathit{\boldsymbol{\theta)}}$. Here, $[x]$ represents the maximum integer less than or equal to $x$. When $\beta$ = 1, the discrete Weibull distribution degenerates to the geometric distribution $Geo(q)$ with $q = 1-\alpha$.

Naturally, the cumulative distribution function of $X$ takes the following form:

3.2.   The US algorithm for the MLE of $\alpha$ and $\beta$

This section will go into detail on the application of the US algorithm for maximum likelihood estimation of the Type Ⅰ discrete Weibull distribution. Assume $X$ is a random variable with the Type Ⅰ discrete Weibull distribution $DW(\mathit{\boldsymbol{\theta}})$, where the parameter vector $\mathit{\boldsymbol{\theta}} = (\alpha, \beta)^T$ is in the parameter space $\Theta \subset \mathbb{R}^2$. Let $\textbf{x} = (x_1, ..., x_n)$ denote the observed values of the random sample $(X_1, ..., X_n)$. Then, the log-likelihood function of the parameter vector $\mathit{\boldsymbol{\theta}}$ is given by

First, the first-order partial derivative of $\ell(\mathit{\boldsymbol{\theta}}|\textbf{x})$ with respect to $\alpha$ can be calculated as

Next, we construct the FLB function regarding the parameter $\alpha$:

Then, the US iteration of $\alpha$ can be obtained as follows:

where $C_1 = h_1(\alpha^{(t)})$, $C_2 = \sum_{i = 1}^{n}[4x_i^{2\beta^{(t)}}+2x_i^{\beta^{(t)}}]$, and $C_3 = 3C_2\left[\frac{1}{\alpha^{(t)}}-\frac{1}{1-\alpha^{(t)}}\right]$. Similarly, we can obtain the first-order partial derivative of $\ell(\mathit{\boldsymbol{\theta}}|\textbf{x})$ with respect to $\beta$,

In order to construct the FLB function and derive the US algorithm without explicit solutions for the two parameters, we need the following two lemmas.

Lemma 1. ^[18] Given that $\theta > 0$, we have

Lemma 2. ^[18] For any $\theta\geq0$, we have

We will then build the FLB function with respect to $\beta$. First, we calculate $h_2(\beta)$'s first derivative.

It can be deduced from Lemma 1 and Lemma 2 that

where $I(\cdot)$ is the indicator function. Then, the corresponding FLB function is obtained as follows:

Therefore, the US iterative process for parameter $\beta$ can be given by

where

The algorithm process for estimating two parameters can be described as follows. In the first stage, we determine the FLB functions for parameters $\alpha$ and $\beta$ using Eqs (3.2) and (3.5), respectively. Subsequently, we set two initial values $\alpha^{(t)}$ and $\beta^{(t)}$, calculate Eq (3.3) to get $\alpha^{(t+1)}$, and then compute $\beta^{(t+1)}$ via Eq (3.6) using $\alpha^{(t+1)}$ and $\beta^{(t)}$. If both of the estimates for the parameters satisfy the convergence criteria, then their corresponding values will be returned. Otherwise, we resume to update the iteration value and repeat the preceding steps until the two estimated parameters converge.

4.   Simulation study

In this section, we conduct simulation studies to confirm the applicability to complex nonlinear equations and compare its performance to that of the classic Newton algorithm. First, we provide the calculation steps for parameter estimation using the Newton algorithm as follows:

The sample size of the studies is set as $n$ = (50,100,200), and the parameters are set as $\alpha = (0.2, 0.4, 0.6, 0.8)$ and $\beta = (0.5, 1.0, 1.5, 2.0)$, respectively. We independently generated $X_1^{(k)}, \dots, X_n^{(k)} \overset{iid}{\sim} DW(\alpha, \beta)$, where $k = 1, \dots, K$($K$ = 1000). The MLE of the parameters under the US algorithm were computed via Eqs (3.2) and (3.4). For every combination of parameters, we ran 1000 iterations of the experiments and evaluated the two methods' fitting performance using the convergence percentage and the mean squared error (MSE) of parameter estimation.

Tables 1–4 display the outcomes of the two algorithms' simulations for each scenario. The MSE of the parameters under both algorithms progressively drops as the sample size rises, according to the statistics in the table, suggesting that both techniques are asymptotically unbiased. When the value of $\beta$ is fixed, as the value of $\alpha$ increases, the MSE of $\beta$ will steadily decrease. Overall, the MSE of both parameters under the US algorithm is smaller than that of the Newton algorithm, suggesting that the US algorithm performs better when it comes to estimation. Furthermore, it is evident from the table's convergence percentages that the US algorithm is more stable.

The trend of the predicted values of $\alpha$ and $\beta$ using the US algorithm and Newton technique as the number of iterations grows is shown in Figure 1. From a stability standpoint, the Newton method shows significant instability and often requires multiple twists to preserve the correct trend, while the US algorithm approaches the true values of parameters monotonically. From a convergence speed perspective, the FLB method exhibits linear convergence, while the Newton algorithm demonstrates quadratic convergence. Consequently, the FLB method has a comparatively slower convergence rate.

5.   Applications

This section describes the analysis of two different real data sets to illustrate the applicability of the US algorithm. The first data set in Table 5 contains the remission times in weeks of 20 leukemia patients with treatment studied by Hassan et al. ^[19]. Another data set is from the National Highway Traffic Safety Administration (www-fars.nhtsa.dot.gov) of the United States, which reports the number of fatalities due to motor vehicle accidents among children under the age of 5 in 32 states during the year 2022.

We model the two data sets with the Type Ⅰ discrete Weibull distribution and the geometric distribution. The fitting results for the leukemia patients data by two distributions are provided in Table 6. The results of the Cramer-von Mises test, Anderson-Darling test, and Kolmogorov-Smirnov test show that the two distributions can successfully fit the data set. The p-values from the three tests of the Type Ⅰ discrete Weibull distribution employing the US algorithm demonstrate the best fitting effect. Moreover, the values of Akaike information criterion (AIC) ^[20] and Bayesian information criterion (BIC) ^[21] also show that the DW distribution based on the US algorithm has better estimation effect.

Table 7 shows the results of fitting the vehicle fatality data with the Type Ⅰ discrete Weibull distribution and the geometric distribution. Comparing the p-values of the three tests at the significance level $\alpha = 0.05$ reveals that the Geo distribution and the Type Ⅰ discrete Weibull distribution estimated by the Newton algorithm are considered to be insufficient. The fitting effectiveness of the DW distribution using the US algorithm is significant, as indicated by the values of AIC and BIC. The histogram for two different data sets evaluated by the DW distribution and the Geo distribution is shown in Figure 2. Figures 3 and 4 present QQ plots for these two distributions. It is also evident that the US algorithm performs better.

6.   Conclusions

The US algorithm is a novel iterative method with high stability and convergence. The existing research only involves simple models such as univariate nonlinear equations or univariate functions. This paper extends the US algorithm to more complex cases of two parameter discrete distribution functions, where one parameter cannot be explicitly represented by the other parameter estimate. In order to successfully estimate the parameters, this paper combines the FLB method to perform optimization estimation of a distribution function. The simulation results for the Type Ⅰ discrete Weibull distribution demonstrate that the US algorithm has good accuracy and stability. Simultaneously, for the purpose of demonstrating the applicability of the algorithm in complex situations, this paper conducted empirical research on two real data sets that follow the Type Ⅰ discrete Weibull distribution, namely, the data from patients with leukemia and children who die from motor vehicle accidents. After comparing and analyzing the US method with the conventional Newton algorithm, the results show that the recommended strategy has an excellent fitting effect.

Author contributions

Yuanhang Ouyang: Formal analysis, Writing original draft, Software, Investigation, Methodology, Data curation; Ruyun Yan: Validation, Software, Formal analysis; Jianhua Shi: Validation, Resources, Writing-review and editing, Methodology. All authors have read and approved the final version of the manuscript for publication.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

This research was conducted under a project titled "The National Social Science Fund of China" (20XTJ003).

Conflict of interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.