Special major 1, 3 satisfiability logic in discrete Hopfield neural networks

1.   Introduction

Artificial neural networks (ANN), drawing inspiration from the complex operations of the human brain, have triggered a global revolution in the field of artificial intelligence ^[1]. They act as a powerful tool across various sectors including medicine ^[2], economics ^[3], transportation ^[4], education ^[5], and business ^[6] to facilitate the development of advanced systems to solve complex intelligent computations. These networks, resembling the neurons in the human brain, consist of interconnected neurons that "learn" from examples and generalize beyond learning data. Each neuron acts as a simple processing unit, with inputs weighted to determine network strength ^[7,8]. Following this, the neurons combine all inputs and generate an output, the fundamental principle established by ^[9]. By employing a specific rule or activation function, the neuron produces an output based on the principle of Hebbian learning, emphasizing the significance of recurrent activation patterns in strengthening neural connections. The Hopfield neural network (HNN) is a prominent example of an ANN that was characterized using an associative learning model. Within this network, data is stored using a bipolar representation and linked by synaptic weights. Over time, great improvements have been made to this network to address optimization challenges. Through the implementation of performance metric tasks, ANN practitioners craft the most effective structure for the HNN. Although substantial progress has been made in recent years, still more findings are needed in the optimal configuration of HNN ^[10]. Therefore, identifying the most suitable symbolic arrangement to govern the HNN requires special attention, especially when dealing with limited capacity and addressing the challenge of local minimum energy in resolving complex optimization problems.

Boolean satisfiability (SAT) is known as the task of determining a configuration that fulfills a given Boolean formula ^[11,12]. A Boolean formula involves operations with Boolean variables that result in outcomes of either TRUE or FALSE, often represented as {–1, 1}. For example, as described by ^[13], a conjunctive normal form (CNF) formula is composed of multiple clauses. Every clause is composed of a set of combined literals. The satisfaction of a CNF formula depends on setting the Boolean variables in a manner that satisfies all the clauses simultaneously. In their study, ^[14] utilized the highest capabilities of the discrete Hopfield neural network (DHNN) to enhance a structured SAT logical function referred to as 3-satisfiability (3SAT). In this specific logical rule, each clause consists of exactly three literals. This study explores pattern satisfiability with a specific focus on 3SAT and the model integration with DHNN. Furthermore, in their study, ^[15] introduced an evolved iteration of 3SAT called maximum k satisfiability (MAXkSAT) within the structure of the DHNN. The presented research was enhanced through the application of the clonal selection algorithm derived in the DHNN, achieving a notably superior global minima ratio. However, in the retrieval phase of the DHNN, the inclusion of non-redundant literals in MAXkSAT led to imperfect outcomes. The introduction of various non-systematic logical rules that allow for different orders and systematic logical rules is being overtaken by their non-systematic counterparts in dominance. Continuously, the research by ^[16] utilized a hybrid technique to optimize the 2SAT logical rule with DHNN. It is important to note that DHNN was used to minimize logical inconsistencies when interpreting the logic clauses. The introduction of 2SAT brought about a revolution in logical rules, giving way for the development of maximum 2 satisfiability (MAX2SAT). This new framework adopts a systematic logic structure with negative literals, giving exciting possibilities and outcomes. Additionally, it is interesting to note that the DHNN has the remarkable ability to reproduce MAX2SAT behavior across various performance measurements. These growths are truly revolutionizing the field of systematic logic. The work by ^[17] presented noteworthy research that explores the implementation of the 3SAT logical rule in DHNN. This work is part of a modified clonal selection algorithm (CSA) to accomplish the effectiveness of 3SAT by successfully evaluating real data sets using the reverse analysis method. This demonstrates the power of systematic SAT in accurately representing and analyzing real-world data sets. The strategy of 2SAT ^[18] incorporated a mutation operator within the retrieval phase of the DHNN. This new method adds another layer of complexity and effectiveness to their methodology. The exhaustive search (ES) algorithm indicates that utilizing an optimizer during the retrieval phase can greatly enhance the ability to uncover additional solutions within various search areas. Based on the observations from the previous study, there is a very negative impact on the global minimum ratio when the number of neurons increases in the systematic logic or the non-systematic logic. Therefore, structuring the logical rules is a crucial method to get a global solution to the network. The logical rule may be systematic or non-systematic with clauses that contain positive or negative literals, which can have a huge impact on the network.

The new exploration by ^[13] proposed using logical structure as a paradigm for DHNN instead of a tool for optimization. Logic was implemented due to the ability of the database capacity to explicitly state information with the systematic approach and incorporate resolution techniques. These attributes greatly assist the DHNN in providing evidence and challenging the desired objective of the network. The integration of logic as a symbolic rule within DHNN has paved the way for the development of various other representations of logic. One noteworthy example is the Abdullah method ^[13], which enables the identification of the optimal synaptic weight linked to the embedded logical rule. The Abdullah method shows how logic-driven approaches continue to advance and enhance neural network technologies. In their study, ^[19] introduced the innovative use of 2SAT logic in DHNN. To enhance the efficiency of 2SAT, they cleverly utilized a genetic algorithm during the learning phase. This optimization technique proved to be highly effective in their research. The study of ^[20] developed a unique approach called random 2 satisfiability (RAN2SAT) that combines first-order and second-order clauses as non-systematic logical rules. Another exploration in systematic logic called weighted random 2 satisfiability (r2SAT) in DHNN was a concept introduced by ^[21]. The involvement of the logic phase in this logical structure ensures that the appropriate ratio of negated literals is imposed on RAN2SAT. This innovation has significant implications for enhancing problem-solving capabilities. However, the binary neurons in a DHNN may offer simplicity but it is also important to note that the use of continuous values within the range of –1 to 1 can sometimes result in precision issues and inaccuracies.

The research by ^[22] embarked on an interesting study. The study discovered ways to expand and explore more solution spaces in neural networks. Building upon this work, ^[22] went a step further by introducing a third-order clause. This allowed for the creation of higher-order non-systematic logic. This new research opens up exciting possibilities for advancing the field of logic into DHNN. The exploration into the elaboration of a novel logical rule known as random 3 satisfiability (RAN3SAT) merges 3SAT logic with the established RAN2SAT concept. This exploration represents an exciting leap forward in this field of research. The results of the integration of RAN3SAT into DHNN have led to the exciting discovery of surpassing RAN2SAT when the RAN3SAT increases toward the ratio of global minima. This finding highlights the superior performance of RAN3SAT in this regard. However, it is important to note that RAN3SAT struggles with the capacity to handle a lower number of neurons. To address this issue, ^[23,24] proposed a revolutionary election algorithm (EA). This algorithm was specifically designed to enhance the learning phase of RAN3SAT by optimizing the model for maximum efficiency and effectiveness. By incorporating the EA into the process, they were able to overcome the limitations previously faced in this area of study. This incorporation enables the DHNN to achieve an optimal final neuron state with higher neuron variation and a low similarity index. The EA enhances the storage capacity of the DHNN through a non-systematic logical rule. Another study by ^[25] has introduced a variant in logical structure called major 2 satisfiability (MAJ2SAT) leading to further advancement in logical rules. This novel approach incorporates a significant proportion of 2SAT clauses relative to the total number of clauses that enhanced problem-solving capabilities. The increased presence of 2SAT clauses in non-systematic logical rules significantly impacts the final neuron state, creating a wider dimension of possibilities. In this regard, MAJ2SAT surpasses the current systematic logical rule by achieving a higher ratio of global minima and diversity while utilizing an equivalent number of neurons. This demonstrates the superior performance and efficiency of MAJ2SAT in comparison to its counterparts. In a recent study, ^[26] explored stimulation data in the domain of non-systematic logical rule structures. They expertly merged the characteristics of systematic and non-systematic logic to introduce Y random 2 satisfiability (YRAN2SAT). This novel approach offers promising new insights and possibilities in the field of logical rules. This method combines logical rules with symbolic instruction, significantly improving the effectiveness of the DHNN. The advantage of both these techniques is that they offer unparalleled benefits and enhance the overall performance of DHNN. In their research, ^[27] introduced a new higher-order non-systematic logic called G random k satisfiability (GRAN3SAT). GRAN3SAT randomly generates a set of clauses of first-, second-, and third-orders. The integration of GRAN3SAT into a DHNN shows the wide storage capacity and capability to handle difficult dimensional problems. Another innovative non-systematic logic developed by ^[28] in the S-type random k satisfiability model introduces a unique approach by assigning negative literals in the logical structure. This method relies on statistical parameters to ensure an accurate and efficient neural network. The introduction of a probability-based system enhances the overall logical analysis process to improve the results. This integration aims to reduce logical rules associated with the zero-cost function. The S-type random k satisfiability also involves identifying the synaptic weight configuration of the DHNN that yields a cost function equivalent to the fulfilled δ2SAT. While new approaches in non-systematic logic have been developed to explore the storage capacity of neural networks, there is little research being conducted utilizing major higher-order clauses within neural networks. These major higher-order clauses have the potential to significantly expand the search space, allowing for increased storage capacity.

In this research study, we propose the application of major higher-order clauses instead of first-order clauses, which has not been thoroughly explored in previous literature. The performance of the DHNN in terms of the combination of major higher-order and minimum lower-order clauses has received limited attention. This research aims to examine how the integration of major higher-order and minimum lower-order clauses can help reduce errors in both the learning and retrieval phases. By examining the impact of this integration, the study seeks novel findings on a potential solution for reducing errors and improving overall performance. This research holds significant promise for enhancing the effectiveness and accuracy of future learning and retrieval processes. The major random 1, 3 satisfiability (MR1, 3SAT) is an innovative approach that combines major higher-order and lower-order non-systematic logical rules within the DHNN. By utilizing the higher-order logical rule, MR1, 3SAT achieves wider storage capacity. The introduction of MR1, 3SAT aims to significantly enhance the storage capacity of DHNN by carefully combining higher-order and lower-order clauses. This research seeks to understand the impact of this integration on reducing errors during both the learning and retrieval phases of the network. By presenting novel findings in this unexplored area, we contribute valuable insights that can potentially revolutionize the effectiveness and accuracy of future learning and retrieval processes by offering an outperforming solution to improve the overall performance in DHNN. On the other hand, the lower-order logical rule aids in making precise predictions and classifications. Our work represents an initial effort to combine clauses as a logical rule within a DHNN to contain all previously proposed sets of logical rules. The main objectives of this study are stated below:

(a) To formulate a new non-systematically logical rule named MR1, 3SAT by combining major higher-order with minimum first-order logical rules into a single formula. This logic involves the selection of major third-order clauses based on a ratio and accommodates non-redundant variables.

(b) The implementation of MR1, 3SAT in a discrete Hopfield neural network involves finding the minimum cost function of the sub-logical rule that is satisfiable. In this approach, neurons are represented as a literal in the logical rule, and their optimal synaptic weights are determined by comparing the cost function with the Lyapunov energy function.

(c) To evaluate the performance of the proposed hybrid network, we use simulated datasets. This hybrid network comprises MR1, 3SAT, the ES algorithm, and the discrete Hopfield neural network. This method will evaluate how this logical structure is able to perform well with less error in the learning and retrieval phase.

(d) To compare the performance of the newly proposed MR1, 3SAT approach with the existing non-systematic logical rules during both the learning and retrieval phases, the similarity index is used to measure the lower similarity between the final neuron and the benchmark neuron state.

The organization of this paper is outlined as follows: Section 2 presents the motivation for this research. Progressing to Section 3, we delve into the complete integration of MR1, 3SAT into a DHNN. The specifics of the experimental setup and the metrics utilized to assess performance throughout the simulation are expressed in Section 4. The exploration of the behavior and effectiveness of DHNN-MR1, 3SAT across parameters and stages along with a comparative analysis against established logical rules is conducted in Section 5. Finally, Section 6 contains the discussion of the results obtained in this research, and Section 7 concludes and discusses the future work of this research.

2.   Motivation

In this section, we will discuss the motivation behind our work. Each motivation addresses the new exploration of existing works and how the proposed logical structure can fill the gaps in the field. This study aims to introduce a new way of organizing or understanding information in the field of discrete Hopefield neural networks.

2.1.   Exploring the adaptability of non-systematic logic

The establishment of the logical rule facilitates the transition of DHNN neuron performance from the initial state to the final state. The absence of this rule exposes the DHNN to the risk of becoming trapped in a cycle of trial and error, lacking a clear bias toward converging into a state of absolute minimal energy. In such scenarios, the initial neuron state attained through a specific DHNN configuration presents a potential solution to an optimization problem. Previous research by ^[14,15] has effectively integrated the k-satisfiability logical rule into the DHNN. Nevertheless, the final neuron state in these engages by a fixed number of dimensions per clause. To illustrate, ^[14] generated a final neuron state that approaches a hundred percent of the local minima ratio. However, their approach limitation is two dimensions, which restricts the formulation variables in defined clauses and consequently limits the capacity of the logical rules. This constraint becomes more visible when higher-order logical rules as demonstrated by ^[15] are integrated into the DHNN. The satisfiability of the logical structure increases when the tendency of neuron connection across different clauses rises in the overall neuron count.

Following the findings of ^[26], they introduced a novel strategy of introducing random second-order and first-order clauses to assist issues related to overfitting. Their methodology diverges notably from that of random k satisfiability (RANkSAT), which imposes constraints on clause utilization. Similarly, ^[27] follows a comparable approach by incorporating randomly generated third-order, second-order, and first-order clauses within the DHNN to amplify the adaptability of the logical framework during the retrieval phase. In our current study, we present a logical rule that directly addresses these limitations. Our approach entails the utilization of a major random generator which produces third-order clauses as well as minimal first-order clauses. This innovation empowers the DHNN to proficiently represent non-systematic logical rules that effectively transcend the previously mentioned problems.

2.2.   Restricted storage capabilities

In DHNN the higher-order logical rules take into account the connections between three variables at the same time that become better at representing complex patterns. The increased logical structure capability has the potential to enhance performance in tasks that demand the capturing of complex relationships. Non-systematic logic splits structurally from systematic logic by accommodating fluctuations in the number of variables per clause randomly upon the specific logical formula. Typically, non-systematic logical rules can be classified into two overarching viewpoints, utilizing first-order clauses and not involving first-order clauses to express the rules. However, the involvement of first-order clauses within non-systematic logic has been identified as presenting challenges when it comes to achieving satisfaction. This is because of some limitations of the interpretations available which lead to the logical rule containing first-order clauses to minimize the logical rule cost function. For instance, an illustrative case is RAN2SAT ^[24], which integrates first-order clauses. This particular approach was deemed ineffective during the learning phase as it requires a greater number of iterations to attain the desired cost function. When the HNN fails to reach the absolute minimum cost function during the learning process, the network could potentially retrieve inaccurate synaptic weights. Consequently, this increases the probability of becoming trapped within local minimum energy states. Conversely, the second perspective of non-systematic logic entails the exclusion of first-order clauses that bears a direct connection to the principal logic.

The authors in ^[22] proposed a variant of RAN3SAT that combines second-order and third-order clauses. This new logical formulation was successfully integrated into the HNN, which resulted in the highest global minima ratio among all RAN3SAT variants. The introduction of third-order clauses into the formulation aids in augmenting the storage capacity of the HNN. As a result, the DHNN exhibits heightened efficiency in recovering neuron states across various logical orders. While the capacity to satisfy the cost function with potential interpretations surpasses that of the non-systematic logic of the RANkSAT, accurate interpretations remain confined to the predetermined count of first-, second-, or third-order clauses allocated to the logical formulation. Consequently, this situation prompted our study to adopt the strategy of randomly generating major third-order clauses. This approach aims to address the limitation observed in RANkSAT, which confines the storage capacity for retrieving diverse solution states that fulfill the cost function. This complication can be effectively addressed by proposing a different order logical rule that includes both third-order and first-order clauses, thereby enabling the retrieval of an expanded range of final neuron states to achieve the global minimum energy.

2.3.   Performance of major clauses

The work ^[25] introduced MAJ2SAT clauses integrated into the DHNN as logical structure MAJ2SAT. The aim of implementing MAJ2SAT is to emphasize the incorporation of more features from 2SAT rather than 3SAT into DHNN. By constructing an efficient MAJ2SAT, we can expand our exploration of a broad solution space. In another attempt, ^[29] introduces major 3 satisfiability by exploring the previous work by ^[30]. Their study incorporates a multi-objective election algorithm into the learning phase of major 3 satisfiability logic. This addition aims to improve the learning process by utilizing the exploration and exploitation mechanism while replacing the ES. This leads to the major higher-order clauses having the potential to increase the storage capacity, accuracy, and diversity of DHNN. To achieve this, it becomes imperative to contemplate the integration of higher content addressable memory (CAM). The CAM is greatly influenced by the availability and quality of the retrieval phase. It is crucial to have a sample and accurate final neuron state to achieve optimal results. Sufficient and diverse learning examples that cover the range of logical rules and their combinations can help the network learn effective weight configurations. Adequate representation of both major third-order and lower first-order clauses in the retrieval phase is crucial for the network to generalize well to the unseen behavior of the final neuron state.

3.   MR1, 3SAT logic

The major random k satisfiability (MRkSAT) is an incredibly useful non-systematic logical structure presented in the easily understandable CNF. It consists of a sequence of clauses with random literals, and the numbers of clauses and literals are randomly determined. MRkSAT is mainly comprised of k-SAT, where the value of k is either 1 or equal to 3. In the context of k-SAT, there is a collection of p literals and q clauses. Each literal in this context can have a value of either TRUE or FALSE, denoted as [–1, 1]. The well-defined structure of MR1, 3SAT is as follows:

(a) A set of p variables, ${A_1}, {A_2}, {A_3}, ...., {A_n}$.

(b) A set of clauses, denoted as Z, is defined as $Z\, = \, \left\{ {{N_C}_1, {N_C}_2, {N_C}_3, ..., {N_C}_t} \right\},$ whereby

where ${m_i}$ represent the number of the third-order clauses, and ${n_i}$ represents the number of the first-order clauses.

(c) The third logic clause is represented as $q = \, \, {m_i}\, + \, {n_i}, \,$where ${m_i}\, \, > \, {n_i}$.

(d) A set of literals, where each literal can be either a variable A or the negation of variable $\neg A.$

(e) The number of q distinct clauses presented, interconnected by the logical AND (∧) operator. Each clause within this collection is composed of precisely three literal variables forming the basis of the k-SAT clause, and in which each ${m_i}$ consists of exactly three variables forming the k-SAT clause, and the ${n_i}$ consist of first-order clauses that every logical clause normally has exactly k variables that are linked with the OR ($\wedge$) operator. The general formula for the MR1, 3SAT expression is

where

Where $C_i^k\, \, \in \, \, \left\{ {{A_i}, \neg {A_{i, }}\, {B_i}, \, \neg {B_i}, \, {C_i}, \, \neg {C_i}, \, } \right\}$ and $\, C_i^k\, = \left\{ {{D_i}, \, \neg {D_i}} \right\}$ are variables in ${L_{MR1, 3SAT}}$. The $C_i^3$ refers to the third-order clauses, and the $C_i^1$ refers to the first-order clauses. A possible example of the formulation of the logical structure given with the ${m_i}$ more than 0.7 is

As presented in the above equation, ${L_{MR1, 3SAT}}$ is satisfiable when $\left({{A_i}, { }{B_i}, { }{C_i}, { }{D_i}} \right)$ in the initial neuron state is $\left\{ { - 1, 1, 1, 1, - 1, 1, 1, 1, - 1, 1, 1} \right\}$, which represents true, or when the $\left({{A_i}, { }{B_i}, { }{C_i}, { }{D_i}} \right)$ initial neuron state is $\left\{ { - 1, 1, 1, 1, - 1, 1, 1, 1, - 1, 1, 1} \right\}$, it is not satisfied. The major third-order logical structure does not consider redundant literals, as it is not satisfied when the initial neuron state represents true.

4.   MR1, 3SAT in the discrete Hopfield neural network

A DHNN is a computational model consisting of interconnected neurons without any hidden layers. Unlike traditional neural networks, the neurons in a DHNN are updated asynchronously, meaning they are updated one at a time rather, simultaneously. This asynchronous updating approach helps eliminate the possibility of neuron oscillations, ensuring stable and reliable computations ^[25]. The neurons in DHNN are represented in bipolar values of –1 and 1 ^[19]. The neuron updates in the DHNN are as follows:

Here, ${S_i}$ is represents the neuron state, while the ${W_{ijk}}$ is the synaptic weight from neuron state i to k, and the $\partial$ is the threshold value of the neuron state. The DHNN strength lies in its parallel computing capabilities and quick convergence, which can exhibit an effective capacity for CAM. This property makes DHNN suitable for tasks that involve pattern recognition, associative memory, and information retrieval. The synaptic weight does not have any interconnection with others, i.e., ${W_{ijk}} = {W_{kij}} = 0$. When the first-order clauses connection is added to Eq (8), any similar neuron connection will result in a synaptic weight of zero. ${L_{MR1, 3SAT}}$ is assigned as a logic into the DHNN as each neuron is a variable. The variables in the ${L_{MR1, 3SAT}}$ will be randomly applied as a clause in Eq (6) until the total number of neurons is satisfied. The cost functions $\mathbb{C}{L_{MR1, 3SAT}}$ of putting the logical rule into the DHNN are given as below in Eq (9):

where NC is the number of clauses, and m+n is the number of variables in ${L_{MR1, 3SAT}}$. The inconsistency of ${L_{MR1, 3SAT}}$ denoted as per Eq (10):

In order to successfully incorporate ${L_{MR1, 3SAT}}$ into the DHNN model, it is essential to ensure that the associated cost function $\mathbb{C}{L_{MR1, 3SAT}}$ is minimized to zero. The DHNN is designed to search for an interpretation that results in a cost function value of zero. By minimizing the cost function, the network aims to find a solution where all constraints are satisfied, and the objective is achieved. This ensures that the DHNN produces an optimal outcome, providing valuable insights and accurate results. By identifying a consistent interpretation, it becomes possible to determine the optimal synaptic weight for ${L_{MR1, 3SAT}}$ ^[25]. However, if $\mathbb{C}{L_{MR1, 3SAT}}$ is not equal to zero $\left({\mathbb{C}{L_{MR1, 3SAT}}\, \, \ne \, \, 0} \right)$, it indicates that ${L_{MR1, 3SAT}}$ cannot be considered as satisfiable. This leads to the synaptic weight being nearly random. In order to ensure accurate retrieval of the final neuron state in the DHNN, it is crucial to employ effective learning methods during the learning phase with the primary objective of achieving $\mathbb{C}{L_{MR1, 3SAT}} = 0$.

In this study, the weights for DHNN-MR1, 3SAT are obtained using the Wan Abdullah method, which relies on identifying logical inconsistencies ^[13]. Each neuron is assigned a truth value, and the objective is to minimize the cost function by maximizing the number of satisfied clauses. In the retrieval stage, before establishing the final state of the neuron the local field plays a significant role in fine-tuning the obtained output ^[31]. The neurons state is asynchronously modified by applying the local field equation. Eq (11) represents the local field

where ${S_i}$ represents the initial state and updated state of neuron i, and ${W_{ijk}}$ and ${W_i}$ denote the synaptic weights for the third- and first-orders of the DHNN, respectively. The primary objective of incorporating ${L_{MR1, 3SAT}}$ into MR1, 3SAT is to achieve a more final state during the retrieval phase which adheres to various logical rules. For example, Eq (11) is utilized to connect the final neuron state with more third-order clauses than first-order clauses as per Eq (6) or (7). The local field plays a crucial role in determining the effectiveness of the final neuron states generated by the DHNN. Its impact on the overall performance cannot be underestimated. Subsequently, the retrieved final states are interpreted to determine if the final solution is overfit or not. This interpretation is carried out based on the updating equation, which can be described by

${h_i}$ is the local field of the network, and the hyperbolic tangent function (tanh) is applied to ${h_i}$ representing the hyperbolic activation function (HTAF), which is a squashing function that ensures the neural network operates efficiently. During the learning phase, the cost function is compared with the Lyapunov energy function of the DHNN to determine the synaptic weight. Consequently, the magnitude of the final neuron state can be assessed using the Lyapunov energy function. The equation $\Upsilon {L_{MR1, 3SAT}}$, as demonstrated in Eqs (14) and (15), refers to the minimum values from the energy function as per the formula for $\Upsilon _{{L_{MR1, 3SAT}}}^{\min }$ in Eq (15):

After the DHNN-MR1, 3SAT learning phase is finished, the synaptic weights derived from it are applied during the retrieval phase. It is essential to highlight that network relaxation is paramount in reaching an accurate final state ^[21]. As the number of neurons increases, more interconnected neurons are involved in the process of saving and receiving information. However, it should be acknowledged that an inefficient relaxation mechanism can result in numerous local minima solutions. In order to achieve a state of relaxation and stability within the network, the neuron updates are executed utilizing the highly effective Sathasivam relaxation method ^[25]. The exchange of information between neurons is computed using the formula

In Eq (16), $h_i^{new}$ is a new local field after the relaxation, while ${h_i}$ represents the local field of the network and R denotes the relaxation rate. In our research, we utilize a relaxation rate of R = 3. It is crucial to differentiate between the global minimum solution and the local minimum solution to ensure that the convergence property of the proposed MR1, 3SAT is satisfied. Additionally, the performance of the final neuron state must satisfy

where Tol represents a pre-established tolerance value of MR1, 3SAT. Eq (17) determines whether the final neuron state satisfies the constraints of the ${L_{MR1, 3SAT}}$.

Figure 1 illustrates the schematic diagram showing the DHNN-MR1, 3SAT implementation. The diagram is divided into two primary phases: the learning phase, and the retrieval phase. During the learning phase, the random arrangement of clauses for ${L_{MR1, 3SAT}}$ is determined and then translated into a Boolean algebra representation. Each clause within is associated with a neuron. In this phase, the objective of MR1, 3SAT is to assign the neuron states that satisfy the cost function described in Eq (10). By achieving the optimal neuron assignment, we can calculate the optimal synaptic weight which will later be employed in the retrieval phase. The retrieval phase involves utilizing the obtained synaptic weight to retrieve the desired information.

Figure 2 illustrates the methodological flowcharts employed in this paper. The green-coloured border signifies the learning phase processes conducted in this study. The clear presentation demonstrates how MR1, 3SAT is initialized and progresses through the learning phase to attain optimal synaptic weights using the respective equations. Conversely, the red-coloured border indicates the retrieval phase of MR1, 3SAT within the DHNN. This section also distinctly illustrates the sequence of steps involved in achieving minimum energy at the conclusion of the DHNN process in MR1, 3SAT. Algorithm 1 shows the pseudocode for the DHNN employed in the context of MR1, 3SAT in this study.

5.   Experimental settings of DHNN-MR1, 3SAT

This section provides an explanation of the proposed logic output and the assessment through multiple evaluation metrics across all phases. The aim is to verify the efficacy of integrating a statistical parameter into MR1, 3SAT, which is designed for logic generation. Furthermore, detailed explanations are given for the simulation platform, parameter assignment and performance metrics. In all models, the ES algorithm was utilized, and we employed a trial-and-error approach to minimize the cost function ^[25]. Table 1 provides the summary of the parameters involved in the MR1, 3SAT.

5.1.   Exploring simulation data for effective analysis

The experiments were conducted using Dev C++ Version 5.11 and executed using open-source software and a 64-bit Windows 10 operating system. To ensure unbiased interpretation of the findings, the simulations were performed on a single personal computer with an Intel Core i5 processor. To ensure impartiality, the same medium specifications were employed. Figure 2 illustrates each configuration of DHNN-MR1, 3SAT, where the green blocks denote the learning phase, while the red blocks signify the retrieval phase.

Third-order clauses in the DHNN play a vital role in capturing complex non-linear relationships among variables which extend two-way interactions. These clauses facilitate the high capacity of complex data patterns that can enhance the precision of models ^[33]. These third-order clauses introduce heightened interactions between three variables, which introduce an augmented realm of logical combinations and relationships. Consequently, the network of logical structure becomes more complex by involving a more advanced stimulation strategy. Nonetheless, the incorporation of major third-order clauses augments network complexity, demanding larger network structures, expanded learning stimulation data and prolonged training durations. The information gathered from major third-order clauses introduces specific solutions by ensemble techniques that involve combining multiple models to improve overall performance without falling into the overfitting trap. The complexity grows in the higher-order model in terms of the interpretability, and is able to reduce unsatisfied models. However, strategies such as error analysis and neuron similarity analysis can predict the major third-order clauses performance in every mechanism in the DHNN.

5.2.   Performance metrics

In this section, we provide an explanation of the performance metrics employed for evaluating the efficacy of the proposed DHNN-MR1, 3SAT in comparison to other established approaches. The performance assessment focuses on two essential phases of the DHNN: the learning phase and the retrieval phase. In the learning phase, the DHNN model aims to minimize inconsistency, which is represented by the network cost function. The retrieval phase of the DHNN plays a vital role in assessing the proposed MR1, 3SAT approach. It consists of evaluating the final state that achieves the absolute minimum energy, and also considers the similarity of the neuron states. Analyzing this phase is essential as it provides a broad understanding of how flexible and effective the proposed MR1, 3SAT.

5.2.1.   Configuring the learning and retrieval phases

This subsection introduces a range of performance metrics for evaluation, which include root mean square error (RMSE), mean absolute error (MAE), sum of squared error (SSE), and mean absolute percentage error (MAPE). These metrics offer robust measures to accurately assess the performance and effectiveness of the proposed models or systems. These metrics have been selected based on previous studies ^[22,25,27] and are deemed relevant for this study. Eqs (18)–(25) represent the performance metrics for both the learning and retrieval phases. A zero value for these errors indicates an optimal learning and retrieval phase. In the learning process, we utilized a string of 130 neurons for learning iterations, as well as the same number of neurons for learning samples. For this process, it is important to note that the learning iterations and the number of learning samples can vary depending on the number of neurons in the neural network, as different numbers of neurons result in different numbers of iterations.

In this analysis, there are eight performance metrics to assess the effectiveness of both the learning and retrieval phase of the suggested network. During the learning phase, we aim to evaluate the network ability to minimize the cost function associated with the logical rule. On the other hand, during the retrieval phase the network is able to achieve the ideal final neuron state through convergence. The RMSE and MAE in the learning phase are used to measure the gap between the maximum clauses and the satisfied clauses in the proposed network model ^[34]. While in the retrieval phase the RMSE and MAE are used to measure the accuracy achieved by the proposed model. The RMSE and MAE in the learning and retrieval phase are described in Eq (18) as

where $\vartheta$ is the number of neuron combinations, ${\Gamma _{\max }}$ refers to the maximum clauses in the model, and ${\Gamma _i}$ represents the total number of satisfied clauses in the model. In the retrieval phase, the RMSE equation is given by

where ${G_{{L_{MR1, 3SAT}}}}$ is the number of global minimum solutions, and ${L_{{L_{MR1, 3SAT}}}}$ is the number of local minimum solutions. The metric ω is the number of combinations, while ρ is number of trials in the DHNN-MR1, 3SAT model, which evaluates the quality of the proposed model. The MAE is a metric defined as the average of the absolute gap between the maximum clauses and the satisfied clauses. The formula for calculating MAE in the learning and retrieval phase is

The MAPE measures the magnitude of the error in percentage in the learning and retrieval phase to evaluate the current performance of the DHNN-MR1, 3SAT model by measuring the difference in the percentage between the maximum number of clauses and the actual satisfied clauses ^[35]. The formula of MAPE in both phases is given by

The SSE used in this research evaluates and quantifies the overall error or discrepancy between the maximum number of clauses and the total satisfied clauses by the proposed DHNN-MR1, 3SAT model. The SSE provides a single and easily interpretable number that represents the total error ^[36]. Equation (24) in the learning phase and Eq (25) in the retrieval phase are used to calculate the SSE in this work.

5.2.2.   Setting for energy analysis

To facilitate the energy analysis, we have employed the approach of ^[17,30], utilizing both global minima and local minima ratios. In the retrieval phase the energy minimization achieved by DHNN-MR1, 3SAT was recorded as similar to the previous performance metrics. The energy analysis can be determined by utilizing Eqs (26)–(28).

5.2.3.   Setting for similarity index

As stated in ^[37], when the DHNN has an initial neuron state, it may lead to bias in the retrieval phase. This bias emerges due to the DHNN learning to memorize the ultimate neuron state outright. To tackle this concern, we can implement a potential solution by following a systematic approach. This involves generating all neuron states using Eq (29), which helps minimize any potential positive (PP) and potential negative (PN) biases. The similarity index serves as a metric for quantifying the correlation between the neurons ultimate state and the desired neuron state in the retrieval phase of MR1, 3SAT. The definition of the optimal neuron state, denoted as $S_i^{optimal}$is

The equation includes A, which represents the positive literal, and $\neg A$, which represents the negative literal present in every clause of MR1, 3SAT. Thereby, $S_i^{optimal}$ represents the ideal neuron state. It is important to note that Eq (29) takes into account the final neuron state that achieves the global minimum energy. In this context, the quality of the final neuron state will be evaluated using the Jaccard index, ${S_{Jaccard}}$ ^[24]

Moreover, it is crucial to note that during this experiment, the instances of false positives and false negatives may result in a trap in the local minimum ratio ^[38]. As a result, an examination of the similarity index (SI) defined in Eq (30) will be conducted. Random clauses and literals will be generated in ${L_{MR1, 3SAT}}$ logic. Additionally, Eqs (31) and (32) represent the total variation (TV) of DHNN-MR1, 3SAT. As mentioned previously, the similarity analysis will be a comparison between the final neuron states obtained and the reference neuron states listed in Table 2.

5.3.   Comparison of method and baseline models

Given that the main objective of this paper is to evaluate the performance of the logic structure by measuring the effectiveness of major third-order clauses, in order to assess this experiment, a comparison is made with existing logics in the DHNN. This comparison focuses on three aspects: logic structure, retrieval phase and quality of the solution. By examining these behaviors in the logic, we can gain insights into their effectiveness and determine their impact on the overall performance of the system ^[39].

To examine the impact of controlling a majority of clauses in the third order and non-systematic logic structure, we investigate how manipulating the majority of clauses influences the behavior and performance of the logic. To evaluate the capability of the logical rule in controlling lower order clauses and accurately reflecting the behavior of the dataset, we aim to determine how effectively major clauses can control the lower order clauses and ensure that the logic aligns with the behavior of the given dataset. By conducting these investigations and comparisons, we can gain insights into the performance and effectiveness of the non-systematic major higher-order logical structure behaviors.

After the implementation of MR1, 3SAT in the DHNN, it is necessary to assess its performance by comparing the quality of the final neuron state with baseline models. Furthermore, an evaluation is conducted to analyze the neuron variations in the retrieval phase, which can identify the presence of global minima solutions and the level of neuron variation. By conducting these evaluations and comparisons, we can obtain a deeper understanding of how well the selected logic in the DHNN performs in comparison to the chosen recent logics with non-systematic structures.

(a) RAN2SAT, as introduced by ^[24], is a logical rule that combines second- and first-order clauses and is implemented in the DHNN as the first non-systematic logic. The logic under consideration, denoted as MR1, 3SAT, exhibits structural differences compared to RAN2SAT while incorporating major higher-order clauses. RAN2SAT is known to provide a wider range of synaptic weights due to the connection of the first-order clause. In this approach, each literal state is randomly defined, but the number of clauses for each order can be predetermined. Specifically, the range of the number of neurons extends from $3 < NN < 50$.

(b) RAN3SAT, by ^[22], extended the previous research by ^[24] by introducing RAN3SAT. This new approach incorporated higher-order logic specifically 3SAT clauses, 2SAT clauses, and first-order clauses within a non-systematic SAT structure. The goal was to address the interpretability limitations of existing non-systematic SAT structures by increasing the number of neurons per clause. While the number of clauses for each sequence was chosen randomly and the literal state was explicitly defined, the number of neurons was limited within a specific range which spanned from $6 < NN < 50$.

(c) RAN3, 1SAT, by ^[22], features different combinations from the RAN3SAT that compared the different order clauses within the logical structure. This innovative approach extended the non-systematic SAT structure by incorporating higher-order logic, specifically 3SAT and first-order clauses exclusively. The primary objective of RAN3, 1SAT was to explore the potential of non-systematic SAT structures by increasing the allocation of neurons per clause. While the number of clauses for each sequence was randomly chosen compared to the MR1, 3SAT, there was a limitation in the selection of clauses. Moreover, the number of neurons of RAN3, 1SAT was limited within a specific range, which varied between $6 < NN < 50$.

(d) YRAN2SAT, introduced by ^[26], is a special logical rule referred to as Y-type random 2-satisfiability. YRAN2SAT stands out due to its unique approach toward randomly generating first- and second-order clauses. It combines both systematic and non-systematic logic that results in a novel framework. YRAN2SAT offers significant flexibility in exploring the search space, allowing for a high potential for diverse solutions by incorporating features from both types of clauses. In this approach, the logical structure defines the total number of clauses while the literal states are randomly assigned. The range of the number of neurons is specified as $1 < NN < 50$, providing a defined scope for the number of neurons in the YRAN2SAT model.

(e) GRAN3SAT, was introduced by ^[27], as an innovative logical framework, referred to as G-type RAN3SAT. This framework entails the random creation of first-order, second-order, and third-order satisfiability logical rules. The involvement of a third-order clause contributes to enhancing the expressive power of this proposed logic. The effectiveness of this logic was evaluated through four distinct simulation scenarios differing in terms of clause counts, ratios of positive and negative literals, learning attempts, and iterations. The range for the number of neurons is specified as $6 < NN < 150$.

5.4.   Benchmark dataset

This paper proposes a model that generates bipolar interpretations from simulated datasets in a random manner with a particular focus on major higher-order clauses. The logical representation employed in the simulations serves as the foundational structure for the generated simulated data. Simulated datasets are frequently employed to model and analyze the effectiveness of SAT logical structure as demonstrated in the works of ^{[22,24,25,27]}.

6.   Results and discussion

In this section, we propose a logical output and measure the effectiveness through the utilization of diverse evaluation metrics in each phase. The objective is to clarify the efficacy of integrating a MR1, 3SAT structure. Furthermore, we delve into discussions concerning the simulation platform, parameter assignments, and metric performance. The simulations were executed with a designated count of neurons denoted as NN. To be precise, the experiments were conducted up to NN = 100 to analyze training error, testing error, and energy analysis. In this case, the similarity index metric placed the simulations at NN = 130 as the simulation reached a 100% local minimum ratio.

6.1.   Learning phase

This section focuses on assessing the errors of all models during the learning phase of the DHNN. In the learning phase, the ES plays a crucial role in facilitating the learning phase by verifying clause satisfaction and the model achieved zero-cost function. This section provides insight into the proposed model synaptic weight management for all logical combinations of MR1, 3SAT.

6.1.1.   Learning error analysis

In this section, we employed RMSE, MAE, MAPE, and SSE as metrics to quantify the error in the learning phase. These metrics evaluate the capability of SAT structures to be learned in the DHNN by assessing the fitness of exhaustive search. Figures 3–6 display the errors attained by various DHNN models in the restricted learning environment with NH = 100. Tables 3–6 shows that the "+" symbol represents the proposed logic MR1, 3SAT surplus when compared to existing non-systematic logic. On the other hand, the "–" symbol represents the existing losses when compared with existing methods. The "Ð" column indicates the difference in error between the proposed MR1, 3SAT and existing non-systematic logics.

ES enables the learning phase to verify the fulfilment of clauses. In Table 3 and Figure 3, it can be observed how the proposed model manages synaptic weights for various logical combinations, including both major the higher-order and first-order clauses. The results indicate that the RMSE learning values were at their lowest when the range of higher-order neural network activations were generated in the MR1, 3SAT formula. This suggests that accurate synaptic weights can be obtained by incorporating more third-order clauses in the MR1, 3SAT logical rules. On the other hand, the MR1, 3SAT model outperformed when the NN increases in the DHNN without required additional learning iterations with the cost function. A low RMSE value can also indicate the ability of the model to avoid overfitting, which means that the model has learned to fit the learning data too closely and generalize to new data. This leads to obtaining optimal synaptic weights.

Table 4 and Figure 4 below show the impressive results achieved by MR1, 3SAT in terms of MAE during the learning phases. It is worth noting that when we minimize MAE learning, we can observe a similar pattern to what has been observed in the existing logical structure. This further highlights the effectiveness and reliability of MR1, 3SAT in producing accurate outcomes. With the help of the DHNN, the ideal final neuron state aligns perfectly with the performance of the MR1, 3SAT model. This dynamic capability ensures that optimal results are achieved, unlocking the maximum potential of the MR1, 3SAT.

Table 5 reveals the MAPE values for MR1, 3SAT and the existing proposed method. In these findings, specifically within the range of $11 \leqslant NN \leqslant 130$, there is an observable trend as the MAPE value increases in the proposed model tends to converge towards 100%. A MAPE value of 100% signifies a notable distinction between the maximum fitness and the achieved neuron fitness. On the contrary, research by ^[40] indicates that a lower MAPE percentage signifies a heightened level of accuracy. Thus, it can be deduced that a smaller MAPE value is indicative of a superior forecasting performance. Figure 5 is a variation graph representing the results from Table 5.

Additionally, Table 6 and Figure 6 highlight a crucial point: the formulation of the SSE learning phase. The values in Table 6 are in the form of logarithms *10,000 for the axis of y. These changes recorded the overall performance errors throughout the learning phase. These results enhance the accuracy and effectiveness of the learning process with the minimum error in the learning phase of the proposed model. The minimum SSE error in the DHNN-MR1, 3SAT learning phase reduced the magnitude of errors.

6.2.   Retrieval phase

This section focuses on evaluating the performance of various logical structures during the retrieval phase of the DHNN. This section is divided into two parts, namely an analysis of retrieval errors, and an assessment of the quality of the final neuron states. The quality analysis includes measures such as total variation and similarity analysis.

6.2.1.   Retrieval error analysis

Retrieval error analysis holds great significance in investigating the behavior of MR1, 3SAT concerning synaptic weight management and the relationship with global or local minima solutions. Once the DHNN completes the verification of clause satisfaction involving the minimization of the cost function, the synaptic weights are generated using the Abdullah method ^[13]. When the cost function reaches zero, it signifies the retrieval of optimal synaptic weights during the retrieval phase, leading to a global minimum solution. Tables 7–10 show that the "+" symbol represents the proposed logic MR1, 3SAT surplus when compared to existing non-systematic logic. On the other hand, the "–" symbol represents the existing losses when compared with existing methods. The Ð column indicates the difference in error between the proposed MR1, 3SAT and existing non-systematic logics.

Based on Figure 7 and Table 7, MR1, 3SAT demonstrated minimal errors in the range of $1 \leqslant NN \leqslant 130$. Meanwhile, during this specific interval, the GRAN3SAT is also in its retrieval phase and shows excellent performance by generating global minimum solutions with the proposed logic.

When a small number of neurons is used for learning and retrieval, the model experiences underfitting during the learning and retrieving of the neuron variations. This is why the model needs to increase the number of neurons. Interestingly, from the retrieval errors it can be observed that, after NN = 40, there were also lower RMSE retrieval errors in the corresponding proposed model compared to the existing models. This is proof that the existence of major order third-order clauses is able to retrieve the ideal final neuron state by exploring more of the solution space.

Conversely, in the range of $1 \leqslant NN \leqslant 30$, MR1, 3SAT experienced an optimal retrieval phase in Table 8 and Figure 8 that shows the results in the lowest errors compared to existing logical structures. The results can assess the optimal and suboptimal retrieval phases by examining the number of global and local minima solutions generated by the proposed model. In summary, when considering all the non-systematic logic discussed, the combinations of RANkSAT exhibit a similar energy profile characterized by a continuous decrease toward the equilibrium neuron state ^[41]. This suggests that at $NN = 130$, the proposed model generates a comparable number of global minima solutions.

The choice of the search algorithm plays a critical role in determining the quality of the solution for synaptic weight management. However, it is important to note that the "trial and error" nature of ES can impact the minimization of the cost function for such searching algorithms ^[18]. If ES fails to identify the optimal synaptic weight, it may have a negative impact on the retrieval phase, potentially leading to a local minimum solution. Alternatively, increasing the number of learning iterations can assist the proposed MR1, 3SAT in achieving an optimal retrieval phase its shown in Figures 8–10. This is because a higher number of iterations has the potential to yield a global minimal solution. One might question why the local minima solution is considered "bad" in the approach. In our simulation, the local minima solution is deemed insignificant because a higher number of local solutions can disrupt the measurement of the similarity in the final states of the neurons. Therefore, this experimental setup aids the network in avoiding several local minima solutions.

It is worth noting that different combinations with non-systematic logical structures yield varying results. Specifically, RAN2SAT exhibits the highest errors compared to other combinations. Conversely, GRAN3SAT demonstrates greater stability and produces fewer errors in terms of synaptic weight management in the $1 < NN < 30$only. This discrepancy can be attributed to the logical structure of SAT with different orders. The involvement of third-order logic proves to be the most effective combination as the probability of obtaining a satisfied interpretation for third-order logic is higher than that of first-order logic. On the other hand, the logical structure of first-order logic can disrupt the process of retrieving correct synaptic weights, which can consequently result in higher retrieval errors. In fact, the MR1, 3SAT restricted the first-order existence clauses.

6.3.   Energy analysis

This section delves into a detailed exploration of the energy profile and the types of solutions, whether global or local, generated by MR1, 3SAT. Tables 11 and 12 shows the analysis of the energy profile involved examining the differences in energy by comparing the values of RMSE, MAE, MAPE, and SSE between the minimum energy and the final energy. These comparisons can be observed in Figures 11 and 12. As the NN increases for all non-systematic logical structures, the number of global minimum solutions decreases. In the minimum energy analysis, it is important to note that logical structures with more literals correspond to the presence of more clauses and require a greater number of iterations to generate feasible solutions. Figure 11 provides further insight demonstrating that higher-order logical structures with major clauses achieve more consistent global minima solutions. Another perceptive is that the absence of logical structures containing first-order clauses have the lowest probability of obtaining satisfying interpretations compared to randomly selected clauses in non-systematic logic.

This indicates that non-systematic logical structures are more susceptible to neuron oscillations. Additionally, an interesting finding is observed in Figure 11, where within the range of $1 \leqslant NN \leqslant 50$we see that MR1, 3SAT can retrieve the highest number of global minimum solutions without any difference in energy. This can be attributed to optimal synaptic weight management, which facilitates the optimal retrieval phase and ensures consistent final neuron states. The significance of the energy profile is to know how well the model has performed in the neural network. The major higher-order clauses significantly improve the solution space in the neural network and is able to store a greater capacity.

The findings of ^[18] suggest that the Lyapunov energy function is bounded and plays a crucial role in determining the dynamics of the Hopfield neural network. In the case of DHNN-MR1, 3SAT, the energy function serves as an indicator of the optimality of the produced solutions. This finding is consistent with the research conducted by ^[42], which explores the quality of solutions in other types of Hopfield networks, including the kernel Hopfield neural network (KHNN) and the mean field theory Hopfield neural network (MFTHNN). According to their study, the Lyapunov energy function is a significant factor in observing the convergence of DHNN. In Table 12 and Figure 13, it can be observed that the SSE _energy increases with NN. This phenomenon occurs due to a lower probability of obtaining a satisfying interpretation for the minimum cost function that causes the higher level of energy. The minimum energy of various order non-systematic logic can be influenced by the existence and manipulation of different SAT clauses which affect the constraints and behavior of the logic circuit ultimately impacting its energy consumption or efficiency.

The findings of this paper suggest that the MR1, 3SAT demonstrates the lowest energy difference among various non-systematic logic orders. This observation may explain why this particular logic is capable of achieving minimum energy compared to other orders. The study employed a bipolar neuron representation with values ranging from –1 to 1. Interestingly, the presence of a zero value eliminates a specific coefficient which can be in zero energy. However, the study emphasizes the significance of the Lyapunov energy function which demonstrates the process of energy minimization by the proposed model. The focus of this study was on major clauses which were restricted in the learning phase. In a similar study conducted by ^[18], multiple DHNN models were examined in a non-restricted learning environment that signified a lower energy profile. Therefore, the conclusion drawn is that a higher number of learning iterations can lead to the attainment of a global minimum energy. It is possible to make modifications to enhance the quality of solutions provided by non-systematic logic in DHNN.

6.4.   Similarity analysis

The similarity index (SI) serves as a metric that allows for the assessment of the similarity or dissimilarity between different data items. The study by ^[24] proposed the use of SI, including the Jaccard index, Sockel Sneat, Dice, and the TV parameter, to evaluate the performance and characteristics of the DHNN model when applied to logical satisfiability. These indexing parameters can help analyze the similarity, dissimilarity, and variations within the neural network that provide valuable information about the effectiveness and behavior of the model in solving SAT instances ^[43].

According to Figure 14, the MR1, 3SAT model exhibits the lowest Jaccard similarity index (JSI) within the range of NN = 1 to NN = 130. This suggests a significant deviation and bias in the generated final states. The elevated JSI value indicates that the model is prone to overfitting since the MR1, 3SAT model is able to produce detectable differences in the final state pictured in Figure 14. The JSI value, which is a measure of similarity, is produced by the MR1, 3SAT model. However, there is no value in between NN = 60 to NN = 100 in RAN2SAT, RAN3SAT RAN3, 1SAT, and YRAN2SAT, indicating that the generated final neuron states vary as the number of neurons increases ^[18]. The decreases in JSI are attributed to the fewer benchmark neurons generated during the retrieval phase by the proposed model.

Beyond that, the JSI decreases to provide meaningful values because all the solutions retrieved by the network are local solutions. This is attributed to the nature of the ES algorithm, which operates based on trial and error and may hinder the minimization of the cost function. As a result, ES fails to produce the optimal synaptic weights during the learning phase, subsequently impacting the final neuron states generated by the model at the end of the computation.

According to ^[44], if the total neuron variation TV increases during the retrieval phase, it could mean that the model is converging well or is experiencing stability. This can lead to avoid overfitting, where the model becomes too specialized to the learning data and does generalize well when the number of neurons increases. In contrast, if the total neuron variation increases during the learning phase, it could indicate that the model is learning to generalize better and is making progress toward minimizing the error in the learning data.

Referring to Figure 15, for NN≥81 it can be observed that the value of TV for MR1, 3SAT reached its lowest point. This indicates that all the final states of neurons converged to local solutions, resulting in a TV value of 0. Furthermore, MR1, 3SAT exhibited the highest TV value before reaching NN≤90 models compared to other existing models when the higher-order clauses were able to retrieve a wider range of solutions. We can see that the existing model GRAN3SAT achieves the highest TV in the smaller NN due to the higher probabilities of the third-order and second-order clauses in the particular model. Conversely, the effectiveness of other non-systematic models declined as the global minimum energy decreased (as shown in Figure 11). Consequently, this leads to a decrease in TV value, and eventually TV reached 0 since there were no final neuron states that achieved the global minimum solution.

7.   Conclusions and future work

In this research, a preliminary analysis examined the effectiveness of a novel non-systematic logical rule called MR1, 3SAT integrated with DHNN. The DHNN-MR1, 3SAT approach possesses several notable strengths, which can be further elaborated upon as follows: First, the combination of first-order and third-order clauses demonstrated that MR1, 3SAT is capable of generating a wider capacity of neuron variations as the ratio of major third-order clauses increase. This observation suggests that the model has the potential to enhance the self-learning process, indicated by ^[22].

Second, the DHNN-MR1, 3SAT approach exhibits the ability to retrieve accurate synaptic weights leading to the attainment of global minimum solutions. This is particularly noteworthy because it remains effective even within a restrictive learning environment. In other words, the model can adapt and find optimal solutions regardless of the limitations imposed during learning. Finally, when comparing different ratios of MR1, 3SAT (with specific total clauses), the results show fewer errors, more stable errors, a greater variety of neuron variations, and overall energy minimization. Specifically, the DHNN-MR1, 3 SAT approach outperformed the existing non-systematic logic that consists of first-order logical combination in terms of error reduction when the number of learning sets was set to 100.

In conclusion, our research highlights the significant strengths of the DHNN-MR1, 3SAT approach. By formulating the combination of third-order and first-order, the model can provide a wider range of neuron variations, enhance the self-learning process, retrieve accurate synaptic weights, and achieve global minimum solutions. Additionally, the major third-order clauses of MR1, 3SAT offer improved error reduction, stable errors, diverse neuron variations, and energy minimization. The novel logical structure MR1, 3SAT existence of the major higher-order clauses hold the potential to advance pattern recognition in real-world data analysis. This advancement can benefit applications in image processing, natural language processing, and other domains that rely on sophisticated data interpretation. It is crucial not only for these applications, but also for sectors such as finance, healthcare, and manufacturing, where rapid and precise decision-making is paramount. These findings contribute to our understanding of efficient logical rule integration and the potential applications in the field.

In future work, with the MR1, 3SAT we will conduct an examination using metaheuristics in the learning and retrieval phases to optimize diversity and achieve global solutions. The application of metaheuristics in both phases is advantageous for generating diverse solutions. During the learning phases, synaptic weight analysis can be employed to address the impact of the energy function and global solutions on synaptic weights. Additionally, according to the existing work proposed by ^[44,45,46], fractional-order systems can capture complex behaviors of the neural network. These studies also proved that integer-order systems could be advantageous in modeling the dynamics of neutral networks.

Notably, the resilient architecture of artificial neural networks combined with our proposed logic provides a solid foundation for practical applications, such as predicting natural disasters. In this context, each neuron represents data attributes such as rainfall trends, river levels, drainage, and ground conditions. These attributes are embedded into the logic-mining approach suggested by ^[32], leading to the development of induced logic with predictive and classificatory capabilities. This logic mining also emphasizes the integration of fractional-order dynamics and bifurcation control mechanisms to enhance the representation of complex dynamics ^[46,47,48] and better capture real-world behaviors within neural networks. We will evaluate their applicability and effectiveness in stabilizing and controlling neural network dynamics considering potential applications in areas such as pattern recognition and information processing.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

This research was supported by Ministry of Higher Education Malaysia for Transdisciplinary Research Grant Scheme (TRGS) with Project Code: TRGS/1/2022/USM/02/3/3.

Conflict of interest

All authors declare no conflicts of interest in this paper.