Online machine learning algorithms to optimize performances of complex wireless communication systems

Koji Oshima; Daisuke Yamamoto; Atsuhiro Yumoto; Song-Ju Kim; Yusuke Ito; Mikio Hasegawa; Koji Oshima; Daisuke Yamamoto; Atsuhiro Yumoto; Song-Ju Kim; Yusuke Ito; Mikio Hasegawa

doi:10.3934/mbe.2022097

Mathematical Biosciences and Engineering

2022, Volume 19, Issue 2: 2056-2094. doi: 10.3934/mbe.2022097

Previous Article Next Article

Research article Special Issues

Online machine learning algorithms to optimize performances of complex wireless communication systems

1.
Innovation Design Initiative, National Institute of Information and Communications Technology, Koganei, Tokyo, Japan
2.
Department of Electrical Engineering, Tokyo University of Science, Katsushika, Tokyo, Japan
3.
SOBIN Institute LLC, Kawanishi, Hyogo, Japan

Received: 31 October 2021 Accepted: 16 December 2021 Published: 27 December 2021

Data-driven and feedback cycle-based approaches are necessary to optimize the performance of modern complex wireless communication systems. Machine learning technologies can provide solutions for these requirements. This study shows a comprehensive framework of optimizing wireless communication systems and proposes two optimal decision schemes that have not been well-investigated in existing research. The first one is supervised learning modeling and optimal decision making by optimization, and the second is a simple and implementable reinforcement learning algorithm. The proposed schemes were verified through real-world experiments and computer simulations, which revealed the necessity and validity of this research.

Keywords:

Citation: Koji Oshima, Daisuke Yamamoto, Atsuhiro Yumoto, Song-Ju Kim, Yusuke Ito, Mikio Hasegawa. Online machine learning algorithms to optimize performances of complex wireless communication systems[J]. Mathematical Biosciences and Engineering, 2022, 19(2): 2056-2094. doi: 10.3934/mbe.2022097

Related Papers:

[1]	Jie Sun, Tzvetanka Boiadjieva-Scherzer, Hermann Kronberger . Incorporation of zirconium into PEO coating on Ti6Al4V alloy from acidic electrolyte. AIMS Materials Science, 2021, 8(6): 974-989. doi: 10.3934/matersci.2021059
[2]	Rafael Resende Lucas, Rita de Cássia Mendonça Sales-Contini, Luis Felipe Barbosa Marques, Jonas Frank Reis, Ana Beatriz Ramos Moreira Abrahão, Edson Cocchieri Botelho, Rogério Pinto Mota . Characterization of the hybrid joint between AA2024-T3 alloy and thermoplastic composite obtained by oxy-fuel welding (OFW). AIMS Materials Science, 2024, 11(3): 585-601. doi: 10.3934/matersci.2024029
[3]	Rucheng Zhu, Yota Mabuchi, Riteshkumar Vishwakarma, Balaram Paudel Jaisi, Haibin Li, Masami Naito, Masayoshi Umeno, Tetsuo Soga . Enhancing in-plane uniformity of graphene nanowalls using a rotating platform for solid-state lithium-ion battery. AIMS Materials Science, 2024, 11(4): 760-773. doi: 10.3934/matersci.2024037
[4]	Shaimaa Mazhar Mahdi, Majeed Ali Habeeb . Low-cost piezoelectric sensors and gamma ray attenuation fabricated from novel polymeric nanocomposites. AIMS Materials Science, 2023, 10(2): 288-300. doi: 10.3934/matersci.2023015
[5]	Bauyrzhan Rakhadilov, Lyaila Bayatanova, Sherzod Kurbanbekov, Ravil Sulyubayev, Nurdaulet Shektibayev, Nurbol Berdimuratov . Investigation on the effect of technological parameters of electrolyte-plasma cementation method on phase structure and mechanical properties of structural steel 20X. AIMS Materials Science, 2023, 10(5): 934-947. doi: 10.3934/matersci.2023050
[6]	Didar Yeskermessov, Bauyrzhan Rakhadilov, Laila Zhurerova, Akbota Apsezhanova, Zarina Aringozhina, Matthew Booth, Yerkezhan Tabiyeva . Surface modification of coatings based on Ni-Cr-Al by pulsed plasma treatment. AIMS Materials Science, 2023, 10(5): 755-766. doi: 10.3934/matersci.2023042
[7]	Min Lai, Xiaofang Yang, Qing Liu, Jinghua Li, Yanhua Hou, Xiuyong Chen, Kaiyong Cai . The surface nanostructures of titanium alloy regulate the proliferation of endothelial cells. AIMS Materials Science, 2014, 1(1): 45-58. doi: 10.3934/matersci.2014.1.45
[8]	Bauyrzhan Rakhadilov, Moldir Bayandinova, Rinat Kussainov, Almasbek Maulit . Electrolyte-plasma surface hardening of hollow steel applicator needles for point injection of liquid mineral fertilizers. AIMS Materials Science, 2024, 11(2): 295-308. doi: 10.3934/matersci.2024016
[9]	Yernat Kozhakhmetov, Mazhyn Skakov, Wojciech Wieleba, Kurbanbekov Sherzod, Nuriya Mukhamedova . Evolution of intermetallic compounds in Ti-Al-Nb system by the action of mechanoactivation and spark plasma sintering. AIMS Materials Science, 2020, 7(2): 182-191. doi: 10.3934/matersci.2020.2.182
[10]	Bauyrzhan Rakhadilov, Rinat Kussainov, Aisulu Kalitova, Zarina Satbayeva, Aibek Shynarbek . The impact of technological parameters of electrolytic-plasma treatment on the changes in the mechano-tribological properties of steel 45. AIMS Materials Science, 2024, 11(4): 666-683. doi: 10.3934/matersci.2024034

Abstract

1. Introduction

Aluminum alloys are widely used in several industrial sectors, from basic products like pots and cutlery to components requiring extreme properties, such as in the case of aeronautical structures and substructures. To meet these requirements, aluminum is processed with the addition of alloy elements such as Cu, Zn, Fe, Ti, Mg, and Mn, giving rise to alloy families like 2XXX, 3XXX, and 4XXX ^[1]. However, alloyed aluminum is susceptible to localized corrosion, known as "pitting", due to the formation of micro-galvanic couples within the aluminum matrix. Additionally, it is characterized by relatively low abrasion resistance and can induce a stress-concentration region when subjected to loads. To mitigate this problem, surface treatments are developed to reduce corrosion and increase scratch resistance ^[1,2].

Aluminum and titanium alloys are also highly versatile and widely used, including in applications such as dental and orthopedic prostheses and aeronautical components (e.g., Ti-6Al-4V), among others. Although titanium is highly reactive, it exhibits good passivation by forming a thin oxide layer, mainly TiO₂ (depending on its chemical composition). However, this passivation is highly dependent on factors such as alloy composition, surface finish, surface area, part geometry, and porosity, among others. This makes the alloy highly susceptible to attacks in environments rich in Cl⁻ ions due to the passive coating's heterogeneous formation ^[3,4,5,6,7].

For this, electrochemical surface treatments have been developed, such as anodization and derived processes, such as plasma anodization, known as plasma electrolytic oxidation (PEO), and micro-arc oxidation (MAO), among others. These treatments aim to develop a thin oxide layer on lightweight alloys, adherent to the substrate, with higher hardness values than untreated material, in addition to improving physical and chemical characteristics ^[8,9,10,11]. Moreover, the PEO process is very similar to conventional electrochemical processes. It should be considered a completely distinct process mainly due to the presence of electrolysis in the aqueous medium by the application of a voltage by power supply and the occurrence of an electrical discharge on the surface's workpiece to be treated (anode). The anodization process and electrical discharge on the material's surface produce oxide coatings with optimized crystalline and morphological structures compared with those produced by conventional processes. The parameters used, such as immersion time, applied potential, current density, temperature, electrolyte composition, and anode chemistry, all strongly influence the coating's characteristics. Additionally, the plasma phenomenon allows the treatment to be carried out in a single step and shorter periods (in the order of minutes). Although the PEO process is widely used in aluminum, titanium, magnesium, and tantalum alloys, there are studies in the literature reporting that it has been applied to alloys that would be considered impossible to treat by conventional methods, such as AISI 1020 steel, copper, and brass ^{[1,9,10,12,13,14,15]}. They are usually carried out with much higher voltages than conventional processes (100–700 V) by using alkaline aqueous solutions based on silicates, aluminates, phosphates, etc.

Given this, the PEO process has been attracting a lot of attention from academia and technology companies, as it is an environmentally friendly process and provides advanced properties to treated materials, ensuring greater added value to new products. Over the last two decades, there has been a significant increase in publications on related topics, as shown in Figure 1.

Figure 1. Number of publications on PEO/MAO (2003–2023).

DownLoad: Full-Size Img PowerPoint

The purpose of this study is to conduct a brief case study presenting the application of conventional anodization treatment on aluminum and titanium parts, comparing them with parts anodized by the PEO process. The aim is to demonstrate to the scientific and industrial community the advantages of replacing conventional processes with plasma treatment.

2. Short literature review

2.1. Aluminum and titanium oxidation mechanism

When aluminum is immersed in an aqueous medium, the metal oxidizes, releasing Al³⁺ ions (Eq 1). Depending on the solution (neutral or slightly alkaline), the reduction of dissolved oxygen occurs (Eq 2).

${Al}_{\left(s\right)}\to {Al}_{\left(aq\right)}^{3+}+{3e}^{-}$

(1)

${O}_{2\left(g\right)}+{2H}_{2}{O}_{\left(l\right)}+{4e}^{-}\to {4OH}_{\left(aq\right)}^{-}$

(2)

With the first two equations, it is possible to derive the overall redox reaction of aluminum (Eq 3) and elucidate the formation of hydrated aluminum hydroxide (Eq 4) due to the reaction between Al³⁺ ions and OH⁻.

${4Al}_{\left(s\right)}+{3O}_{2\left(g\right)}+{6H}_{2}{O}_{\left(l\right)}\to {4Al}_{\left(Aq\right)}^{3+}+{12OH}_{\left(Aq\right)}^{-}$

(3)

${Al}_{\left(Aq\right)}^{3+}+3{OH}_{\left(Aq\right)}^{-}\to {Al\left(OH\right)}_{3\left(s\right)}$

(4)

In solutions rich in Cl⁻ ions, aluminum hydroxide can react to form soluble aluminum chloride (Eq 5).

${Al\left(OH\right)}_{3}+{3Cl}^{-}\to {AlCl}_{3}+{3OH}^{-}$

(5)

Cl⁻ ions accelerate pitting corrosion, as the local region becomes more acidic due to the hydrolysis of aluminum ions ^[16,17,18].

The redox process of titanium is similar to that of oxygen, except for photocatalysis. Initially, titanium oxidizes, releasing Ti⁴⁺ ions (Eq 6), while the reduction of oxygen occurs simultaneously, as shown in Eq 2.

${Ti}_{\left(s\right)}\to {Ti}_{\left(Aq\right)}^{4+}+{4e}^{-}$

(6)

Due to its photocatalytic properties, titanium dioxide (TiO₂) forms electron-hole pairs (Eq 7) ^[19,20].

${TiO}_{2}+hv\to {TiO}_{2}({e}^{-}+{h}^{+})$

(7)

The excited electrons and generated holes can participate in oxidation and reduction reactions. The hole (h⁺) can oxidize water molecules, making the medium more acidic and producing hydroxyl radicals (•OH) (Eq 8). Meanwhile, the electrons (e⁻) reduce the dissolved oxygen (Eq 9).

${H}_{2}O+{h}^{+}\to •\mathrm{O}\mathrm{H}+{H}^{+}$

(8)

${O}_{2}+{4e}^{-}+{2H}_{2}O\to {4OH}^{-}$

(9)

With the formation of Ti⁴⁺ ions (Eq 6) and OH⁻ groups (Eq 9), they react to form titanium hydroxide (Eq 10).

${Ti}_{\left(Aq\right)}^{4+}+{4OH}_{\left(Aq\right)}^{-}\to Ti({OH)}_{4}$

(10)

In studies involving the oxidation of titanium alloy with 99.997% purity and different processing temperatures, Vaquila et al. ^[21] reported that oxidation processes below 200 ℃ resulted in the presence of only the TiO₂ phase on the surface. However, with increasing temperature, other phases were found, such as Ti₂O₃, and for thinner coatings, TiOx (where x < 2).

2.2. Keywords used and selection methods

The Scopus database and Google Scholar were used to compile articles for the present investigation. The principal search terms encompassed plasma electrolytic oxidation, micro-arc oxidation, plasma anodization, conventional anodization, and hard anodization. Articles were chosen based on their discussion and results to facilitate a comparative evaluation with an alternative procedure. To illustrate, research focusing on the "morphological attributes of the 1XXX aluminum alloy subsequent to undergoing treatment by hard anodization" was juxtaposed with those concerning the identical alloy processed through the PEO method, and vice versa.

2.3. Operating parameters and comparison of properties of anodizing and PEO/MAO coatings

As a response to the challenges that metal structures face, including abrasion and corrosion, anodizing and PEO treatments aim to create an oxide layer on the workpiece within an electric field formed during treatment. Relative porosity and crystallinity can be controlled in the process by altering working parameters ^[22].

Among the treatment parameters in both conventional anodizing and PEO processes, immersion time, treatment temperature, voltage, and the electrolyte used can be considered (Table 1). However, when addressing the PEO process, many other parameters benefit the coating properties compared with conventional anodizing, such as duty cycle, frequency, and current density, among others (Table 2).

Table 1. Parameters influencing the properties of anodic coatings.

Parameters	Operational variations	Electrolyte	Substrate	Conclusions	Ref.
Immersion time	40, 80,120 min	C₂H₂O₄	Pure aluminum	The diameter of nanopores increased with the treatment time from 40 to 80 min. However, instability in the nanotubes' walls produced at 120 min resulted in their collapse.	^[23]
	5, 10 h	NH₄F (0.2 M) + H₃PO₄ (0.5 M) + H₂O (distilled) + ethylene glycol	Titanium (99.7%)	With increasing anodization time, the pore diameter and length dimensions increased, reaching 28.7 and 284.6 nm after 5 h, and 30.0 and 376.5 nm after 10 h.	^[24]
	60, 90,120,150 min	C₂H₂O₄ (0.4 M)	Pure aluminum	With increasing anodization time, an increase in the pores' diameter produced was observed, attributed to the rise in local temperature, leading to dissolution of the Al₂O₃ layer and consequently reducing corrosion resistance.	^[25]
Temperature	0, 10, 20, 30, 40, 50 ºC	C₂H₂O₄ and NaOH	Sn	The increase in temperature from 0 to 30 ℃ results in larger and open pores, while temperatures of 40 and 50 ℃ lead to pore sealing. This temperature dependence also affects the coating thickness, as higher temperatures enhance the reaction kinetics of coating growth.	^[26]
	20, 30, 40, 50 ºC	H₃PO₄ (0.4 M)	AA5052	The temperature increase during the anodization process in the H₃PO₄ solution accelerated the barrier layer's formation, reducing the time from 80 s at 20 ℃ to 8 s at 50 ℃.	^[27]
	30, 40, 50 ºC	C₂H₂O₄	Pure aluminum	The temperature solution increase resulted in larger pore diameters and a decrease in their quantity. This also reduced the surface acidity due to fewer transported anions, despite the increase in specific surface area.	^[28]
Voltage	10, 20, 30 V (DC)	C₂H₂O₄	Pure aluminum	Increasing the applied voltage in the solution reduces the surface area, decreases pore density, and increases pore diameter. This leads to higher current per pore and electric field strength, allowing greater alumina deposition.	^[28]
	16, 18, 20 V (DC)	H₃PO₄	AA2024-T3	Anodizing the AA2024-T3 alloy in H₃PO₄ solution revealed that increasing the voltage enhances the oxide layer's thickness. However, beyond 20 V, the thickness decreases due to elevated electrolyte temperature, resulting in the coating's dissolution.	^[29]
	5, 6, 7, 8, 12, 15 V (DC)	H₂SO₄	Pure aluminum	At lower potentials (5–6 VDC), the oxide layer exhibits a thin membrane with smaller pore diameter and lower density. Increasing the voltage (12–15 VDC) enhances both the thickness and pores' density, resulting in a layer with higher void density.	^[30]
Electrolyte	NaOH (1 M), KOH (1 M)	NaOH (1 M), KOH (1 M)	Ti (CP2) Ti6Al2Sn4Zr2Mo Ti6Al4V Ti Beta-C	The NaOH solution yielded better corrosion resistance in Ti alloys due to an increase in impedance.	^[3]
	H₂SO₄ (1 M), H₃PO₄ (1 M)	H₂SO₄ (1 M), H₃PO₄ (1 M)	Ti (CP2) Ti6Al2Sn4Zr2Mo Ti6Al4V Ti Beta-C	H₂SO₄ showed a more homogeneous coating on Ti alloys with fine pores, whereas H₃PO₄ exhibited a coating with larger pores but fewer in quantity.	^[4]
	H₂O + CH₃OH (50%) + HF (1%), H₂O + CH₃OH (90%) + HF (1%)	H₂O + CH₃OH (50%) + HF (1%), H₂O + CH₃OH (90%) + HF (1%)	Pure titanium	With the increase in CH₃OH concentration to 90%, the pore concentration decreases on the surface of Ti alloys, and the current density of the process becomes easily controllable. This makes the process of fabricating standardized nanomaterials better.	^[31]

| Show Table

DownLoad: CSV

Table 2. Parameters influencing PEO coating properties.

Parameters	Operational variations	Electrolyte	Substrate	Conclusions	Ref.
Immersion time	120,210,300 s	Na₂SiO₃ (15 g/L) + Na₃PO₄ (1.5 g/L)	AA2024-T3	The increase in treatment time resulted in the growth of the oxide layer thickness up to 210 s; beyond this point, the coating began to dissolve, leading to a decrease in roughness.	^[10]
	10, 20, 30, 40, 60 min	Na₃PO₄	Magnesium alloy ML-10	Treatment time influences coating thickness, resulting in 10 µm after 10 min and 40 µm after 60 min. Microhardness also increases, ranging from 165 to 360 HV.	^[32]
	60, 90 s	H₂SO₄ (1 M) + H₃PO₄ (1.5 M)	Ti-6Al-4V	The increase in anodization time caused the dissolution of the oxide coating, revealing higher levels of Ti on the surface, while there was a decrease in surface roughness.	^[33]
Temperature	−5, 25 ºC	K₃PO₄ (1 M) + KOH (1.5 M)	Mg AZ31	Room temperature accelerated the nucleation of the coating, while temperatures below 0 ℃ produced a denser coating with smaller pores, improving its corrosion resistance.	^[34]
	5, 15, 25, 35 ºC	Na₂SiO₃ + (NaPO₃)₆NH₄VO₃ + KF + Na-Citrate + NaOH + EDTA (Ethylenediamine tetraacetic acid)	Mg AZ31B	The electrolyte temperature altered the coating's microstructure. As the temperature increased, the pore diameters decreased and their quantity increased, while the coating thickness gradually reduced.	^[35]
	10, 20, 30, 40, 50 ºC	NaAlO₂ + NaOH + Na₂SiO₃	Mg AZ91D	With the increase in electrolyte temperature from 10 to 50 ℃, the surface roughness (Ra) decreased from 0.7 to 0.15 μm, and the corrosion resistance increased from 3.5 to 9 in salt spray tests. The higher temperature also raised the magnesium content in the film from 25% to 63% by weight and reduced oxygen from 66% to 21% by weight, indicating dehydration of the film.	^[36]
Voltage	+400/−0 V, +440/−0 V, +480/−0 V, +500/−0 V, +520/−0 V	Na₂SiO₃ (14 g/L) + KOH (1 g/L)	Pure aluminum	The increase in voltage and the presence of high temperatures in plasma discharge channels lead to oxidation and melting of SiC reinforcement phases, evidenced by the appearance of CO and dissolved Si elements. Additionally, it enhances the decomposition and oxidation process of the electrode.	^[37]
	300,340 V (DC)	NaH₂PO₂·H₂O + Na₂SiO₃·9H₂O	Pure niobium	At 300 V, the formed coating is compact, both on the top and transversally. Increasing the voltage to 340 V results in a uniformly distributed porous structure, with circular pores on the top, leading to an increase in porosity from 2.32% to 26.45%.	^[38]
	350,400,450 V (DC)	Na₂SiO₃ + NaOH + Na₂H₂P₂O₇	AA2024	The coatings obtained at 350 V exhibited micropores with diameters ranging from 0.1 to 1 µm. With increasing voltage, these pores reached diameters of up to 3 µm, accompanied by an increase in length from 0.5 to 5 µm for treatments at 350 and 450 V, respectively.	^[39]
Electrolyte	Ta(OH)₅: 10, 30, 40 g/L	KOH (2 g/L) + Ta(OH)₅ (varying)	AISI 1020	The addition of Ta(OH)₅ to the electrolyte increases the deposition of species in the coating, resulting in the filling of pores and cavities and in the reduction of surface irregularity.	^[13]
	Ⅰ: KOH (2 g/L) + Na₂SiO₃ (18 g/L) + Na₃PO₄ (2 g/L) Ⅱ: KOH (2 g/L) + Na₂SiO₃ (2 g/L) + Na₃PO₄ (18g/L)	Ⅰ: 2K8Si2P Ⅱ: 2K2Si18P	AA2024	In the silicon-based electrolyte, the breakdown voltage was 280 V and the final voltage was 455 V, resulting in a rough surface of 2.6 ± 0.2 μm and porosity of 7.1 ± 1.0%. In the phosphorus-based electrolyte, the breakdown voltage was 260 V and the final voltage was 475 V, with a lower porosity of 4.4 ± 0.3% and smoother surface.	^[40]
	Ce(NO₃)₃: 0.25, 0.50, 1.0, 1.5 g/L	Na₂SiO₃ (10 g/L) + NaOH (2 g/L) + Ce(NO₃)₃	AA6061	With the increase in Ce(NO₃)₃ concentration, the thickness of the ceramic layer gradually increased, micropores and fissures increased, and roughness decreased due to the sealing of some pores.	^[41]

| Show Table

DownLoad: CSV

Tables 1 and 2 provide a comprehensive overview of the properties of oxide coatings produced through the PEO process, showing the interaction of various parameters on surface characteristics, microstructure, and functional properties, such as biocompatibility and photocatalysis ^[10].

Both in the conventional anodizing process and in the PEO/MAO process, the system configuration is an electrolytic cell, where two electrodes are present: the material to be treated on the positive pole (anode) and the cathode, which can be an inert material. In most of the literature, stainless steel and platinum are commonly found as cathode materials ^[1,42,43].

Alloys susceptible to passivation, such as aluminum and titanium, develop a thin oxide layer on their surface, in the order of nanometers, when exposed to oxygen-rich environments. As mentioned earlier, treated metal ions penetrate this barrier layer to form a new oxide layer. The electric field formed during the process surpasses the activation energy of this barrier layer, thus forming a new oxide layer ^[22,44,45]. Usually, for titanium alloys, the required potential is above 100 V (depending on the setup), while for aluminum alloys, Frank et al. ^[24] employed 25V/7A to anodize the AA2024-T3 alloy in a 10% phosphoric acid (H₃PO₄) solution. More recent studies have applied statistical methods to optimize working parameters in both conventional processes and plasma processes ^{[46,47,48,49]}.

The parameter "electrolyte" significantly affects the composition of the generated coating. In conventional anodizing, the electrolytic composition affects the formation and dissolution of oxide films, crucial for achieving the desired surface finish ^[40,50]. In both conventional anodizing and the PEO process, the anion electrolytic incorporation into the anodic film is crucial, with studies showing that it may be necessary for the ignition process in materials such as aluminum, magnesium, and titanium ^[1,50]. Similarly, studies involving adding additives such as ceramic particles, graphene oxide, and organic particles are investigated for various purposes to functionalize the surface ^[51].

The immersion time significantly affects the properties of oxide coatings, both in conventional anodizing processes and plasma processes. In conventional anodizing, longer anodizing times, such as 45 min, result in greater corrosion resistance, improved adhesion, and increased oxide layer thickness, enhancing the overall coating performance ^[47,52]. In the PEO process, immersion time is extremely important as it leads to an increase in pore diameter and a decrease in their quantity, along with an increase in coating thickness ^[42,53].

The processing temperature influences the properties of coatings in both anodizing and the PEO process. In studies applying different temperatures, Guo et al. ^[54] anodized the AA6061 alloy with a sulfuric acid (H₂SO₄) solution. They found that as the temperature increases, the surface changes from a porous structure to a coral-like structure with greater thickness ^[34], conducting a study on plasma anodizing in a Mg alloy, comparing temperatures below zero (268 K) with room temperature (298 K). They found that coatings produced at lower temperatures had fewer cracks, thus improving the samples' corrosion resistance. However, this also resulted in a decrease in coating thickness.

The applied voltage plays a fundamental role in PEO treatment, strongly influencing gas evolution around the anode, the crystalline composition, and corrosion resistance. In studies characterizing the electrolytic plasma, Liao et al. ^[37] found that increasing the voltage in the treatment led to higher concentrations of H₂, O₂, and traces of CO. The concentration of H₂ and O₂ was higher in the silicate solution due to strong sparks in the anode. The applied potential can significantly influence the photocatalytic titanium alloys' properties. For example, increasing the potential causes an increase in the coating's growth rate, as well as an increase in the pore's diameter and the spacing between pores ^[55].

In general, electrochemical treatments significantly modify the surface of metal parts, potentially increasing or decreasing wettability, functionalizing surfaces for several applications, and enhancing wear resistance, among other benefits ^[22,56,57]. In Table 3, brief comparisons of the properties of coatings obtained by anodizing and the PEO process are reported.

Table 3. Properties of oxide coatings by anodizing and PEO ^{[1,10,22,58,59,60]}.

Properties	Anodizing	PEO
Morphology	Porous coating or nanotubes	Highly porous coating
Thickness	To 50 µm	To 300 µm
Crystallinity	Amorphous	Crystalline or amorphous
Hardness	100 to 400 HV	300 to 1500 HV
Adhesion	Low	High
Corrosion resistance	Low	High
Wettability	Hydrophilic	Hydrophilic or hydrophobic
Operational cost	Low	High

| Show Table

DownLoad: CSV

3. Anodization of aluminum: Conventional process vs. plasma process

In a study developed by Lunder, Olsen, and Nisancioglu ^[61] involving the AA6060 aluminum alloy, various surface treatments were conducted to facilitate adhesion using epoxy-based adhesive bonding. The surface treatment, using conventional anodization (CA) in sulfuric acid, allowed obtaining a coating with a thickness of 0.2 µm showing better results and supporting loads between 30 to 35 MPa.

Using the same aluminum alloy AA6060-T6, Shore et al. ^[62] utilized the PEO process with an alkaline solution of sodium silicate (Na₂SiO₃) + sodium pyrophosphate (N₄P₂O₇·10H₂O) and potassium hydroxide (KOH). The parameters used were an anodic voltage of 530 V and a cathodic voltage of 190 V, with a frequency of 2.4 kHz and a duty cycle of 40%. The processes lasted for 3 to 5 min after reaching the specified voltages. After treatment, the samples were bonded with a thermo-polymerizable adhesive "Dow-Betamate 4600F" with a bonded area of 10 × 25 mm and an adhesive thickness of 0.2 mm. Heating was performed in several stages, totaling 60 min of curing, with a final temperature of 190 ℃. The coating had a thickness of 15.0 µm, with globular morphologies with an average diameter of 5 to 20 µm. There were sporadic pores in the coating (relative porosity of 10%), but it allowed good adhesion with the thermo-polymerizable adhesive, with a strength of 34.32 MPa.

It is common for surfaces coated with PEO to exhibit various distinct morphologies including pores, spherical protrusions, and coral-like morphologies. These characteristics vary depending on the electrolyte composition, and when enriched with silicon, it can result in needle-like microstructures known as mullite. This phenomenon leads to a coating's high densification and an increase in surface hardness, making it highly suitable for several technological applications ^{[9,11,12,42,63,64]}.

It is important to emphasize that, despite the similar values obtained in the resistance through the single lap shear test, the PEO process can readily replace conventional anodization methods. This is due to the greater environmental efficiency, resulting from the use of a single alkaline electrolyte, as well as the unified nature of the PEO process, which substantially enhances the physical and chemical coating properties ^[1,12,42,62]. Some works on adhesion between aluminum and composite or aluminum with polymeric adhesives are shown in Table 4, with respective shear values.

Table 4. Lap shear values found in literature.

Materials	MPa	Ref.
Welding with oxy-gas LPG/PEI glass fiber + AA2024 (PEO)	5.2	^[65]
Welding with oxyacetylene/PEI glass fiber + AA2024 (PEO)	10.7	^[48]
Friction injection joining (F-IJ)/AA6082-T6 + PEI glass fiber	1.1	^[66]
Friction stir spot welding (FSSW)/AA5052 (PEO) + polypropylene	1.36	^[67]
Welding by YB-laser in AA2024-T3 (anodized) + PEI glass fiber	16.0	^[68]
Adhesive in AA6060-T6 + AA6060-T6	34.3	^[62]

| Show Table

DownLoad: CSV

Gu, Zhang, and Yu ^[69] treated the AA5052 aluminum alloy by anodization with a mixed acid electrolyte (citric acid + sulfuric acid) to increase the corrosion resistance in simulated acid rain tests. The authors confirmed that with the correct addition of citric acid to the solution, it acts as a corrosion inhibitor, in addition to increasing the thickness of the oxide coating produced.

In a study involving PEO, Lucas, Gonçalves, and Santos ^[1] treated the AA5052-H34 aluminum alloy with a solution containing Na₂SiO₃ and several concentrations of Na₃PO₄ as an additive. It was observed that the crystalline phases (ɣ-Al₂O₃) and mullite tend to increase with the additive's increase, which was corroborated by scanning electron microscopy (SEM) analysis, as shown in Figure 2, highlighting the presence of sealed pores and improving the corrosion resistance. However, there is a limit to the proportion of additives, as an excessive increase results in the presence of amorphous phases, affecting both the morphology and corrosion behavior.

Figure 2. SEM micrograph of AA5052-H34 aluminum alloy plasma-anodized (Reproduced from Ref. ^[1] with permission).

DownLoad: Full-Size Img PowerPoint

In a brief analysis, it can be observed that plasma anodization tends to offer superior corrosion resistance compared to oxide coatings produced by conventional processes. Morphologically, PEO coatings exhibit more closed pores and denser layers. This difference can be compared through data obtained by linear polarization, as demonstrated in Table 5.

Table 5. Corrosion potential (Ecorr) and corrosion current density (Jcorr) of AA5052 aluminum alloy anodized and plasma-anodized AA5052 alloys.

Sample	Ecorr (V)	Jcorr (A/cm²)
AA5052 (anodized)	−0.484	1.6 × 10⁻⁶
AA5052 (plasma-anodized)	−0.120	2.20 × 10⁻⁸

| Show Table

DownLoad: CSV

It is observed that the PEO coating exhibited higher nobility (more positive Ecorr) compared to the coating produced by the conventional process, representing an increase of approximately 75.2%. Considering the corrosion rate in mm/year, according to ASTM G102 standard ^[70], and using Eq 11, the corrosion rate equation becomes:

$CR = \frac{K\times {j}_{corr}\times EW}{D}$

(11)

where, CR: corrosion rate (mm/year); icorr: corrosion current density (µA/cm²); K: 3.27 × 10⁻³ mm·g/µA·cm/year; EW: equivalent weight (9.05 for AA5052); D: material density (g/cm³).

Considering Eq 11, the corrosion rate of AA5052 aluminum alloy anodized by the conventional process would be approximately 1.8 × 10⁻¹⁴ mm/year, while the alloy treated by the plasma anodization process would be 2.4 × 10⁻¹⁶ mm/year, a considerable decrease in the corrosion rate, highlighting the effectiveness of the PEO treatment. These results highlight the significant improvement provided by the plasma treatment, approximately 87%.

It is observed that, due to the morphological characteristics of oxide coating produced by the PEO treatment, it can offer much superior corrosion resistance compared to coatings produced by conventional anodization. This reiterates that plasma processes have the potential to replace conventional methods of surface treatment by conversion, providing more promising results by processing products with higher added value.

The process PEO facilitates the anodization of aluminum alloys (Al-Si), a task known for its difficulty when approached through traditional techniques. Pezzato et al. ^[71] and Valentini et al. ^[72] carried out PEO procedures on the AlSi10Mg alloy, incorporating a range of additives such as molybdenum, manganese, cerium, and graphene. These investigations illustrated notable enhancements in the resistance to corrosion with the inclusion of these additives in the fundamental solution. To clarify the mechanisms by which the additives improved the corrosion resistance of the oxide layer on the AlSi10Mg substrate, the authors provided a schematic representation of the infiltration of the additives into the pre-existing pores post-treatment, depicted in Figure 3.

Figure 3. Representation of the effect of additives on PEO coating: (A) MoO⁴⁻, (B) MnO⁴⁻, (C) CeO₂, (D) TiO₂, and (E) Graphene (Reproduced from Ref. ^[72] with permission).

DownLoad: Full-Size Img PowerPoint

4. Anodization of titanium: Conventional process vs. plasma process

In studies of anodization using NaOH and KOH solution, Tiburcio et al. ^[3] treated different titanium alloys (Ti CP2, Ti-6Al-2Sn-4Zr-4V, and Ti Beta-C), aiming to characterize the electrochemical effect of samples, especially when exposed to environments rich in NaCl and H₂SO₄. The morphological analysis is presented in Figure 4.

Figure 4. Titanium samples anodized: (a) Ti CP2; (b) Ti-6Al-2Sn-4Zr-2Mo; (c) Ti-6Al-4V; and (d) Ti Beta-C (Reproduced from Ref. ^[3] with permission).

DownLoad: Full-Size Img PowerPoint

Tiburcio et al. ^[3] also observed that in the NaOH solution, the TiO₂ oxide coating formation occurred entirely heterogeneously, displaying preferential deposition regions as well as microcracks. They reported that the Ti Beta-C alloy was the only one to show a low rate of oxide coating generation for the medium used. Regarding the coatings' thickness, it was observed that the Ti CP2 alloy presented a thickness ranging from 376 to 400 nm, the Ti-6Al-2Sn-4Zr-2Mo alloy from 1.92 to 2.63 µm, the Ti-6Al-4V alloy in the range of 23 to 25 nm, and finally, the Ti Beta-C alloy exhibited coatings with a thickness ranging from 660 to 756 nm.

In studies involving the titanium alloys VT1-0 and VT5 (Table 6), Ramazanova, Zamalitdinova, and Kovalenko ^[7] treated the alloys by PEO with a pulsed system (anodic/cathodic current: 250 µs/5 ms; frequency: 50 Hz; voltage: 360 to 365 V), with the process lasting 10 min, using three types of electrolytes, as described in Table 7.

Table 6. Chemical composition of alloys VT1-0 and VT5.

Elements	VT1-0	VT5
Ti	99.0% (Min)	Balance
Al	-	6%
V	-	4%
O	< 0.12%	< 0.20%
N	< 0.04%	< 0.05%
C	< 0.05%	< 0.10%
Fe	< 0.08%	< 0.30%
H	< 0.0015%	< 0.015%

| Show Table

DownLoad: CSV

Table 7. Electrolytes used for the treatment of VT1-0 and VT5 alloys ^[7].

Electrolyte	Description	Concentration (g/L)
A	Sodium silicate (metasilicate) 9 aqueous	100
	NaOH	8.0
	Aluminum oxide (1.1–1.5 µm)	20.0
B	Sodium phosphoric acid 3–substituted, 12 aqueous	70.0
B	Aluminum hydroxide (0.6 µm)	20.0
C	Sodium phosphoric acid, two-substituted, 12 aqueous	40.0
	Sodium tetraborate 10 aqueous	30.0
	Boric acid	22.0
	Ammonium fluoride (NH₄F)	10.0
	Aluminum oxide	20.0

| Show Table

DownLoad: CSV

As a result, the authors reported a porous oxide coating, with varied thicknesses for each type of electrolyte and alloy used, as follows: VT1-0 with thicknesses of 5.0, 15.0, and 21.0 µm for electrolytes A, B, and C, respectively; VT5 with thicknesses of 7.5, 9.5, and 19.5 µm for the same electrolytes. The coatings on the alloys exhibited statistically similar thicknesses and relative porosities, with alloy VT1-0 having pores with a mean diameter of 0.4 µm and porosity of approximately 8.7%, whereas alloy VT5 exhibited pores with a mean diameter of 0.4 µm and porosity of approximately 9.2%, as shown in Figure 5.

Figure 5. SEM microscopy of VT1-0 and VT5 samples: (a), (c), and (e): VT1-0 samples, with electrolytes A, B, and C; (b), (d), and (f): VT5 samples, with electrolytes A, B, and C, respectively (Reproduced from Ref. ^[7] with permission).

DownLoad: Full-Size Img PowerPoint

A key observation is that PEO treatments can yield oxide coatings that are thicker and denser than with conventional processes. This is attributed to the plasma properties, which enhance the coating's physical and chemical properties. Additionally, the PEO process offers the advantage of being carried out in fewer steps, resulting in reduced processing time. Furthermore, it is environmentally friendly due to the use of "clean" electrolytes, as reported by several authors ^{[61,62,73,74]}.

To investigate the wear behavior of oxide coatings produced on the Ti-6Al-4V alloy by PEO, Santos et al. ^[75] treated samples using the PEO process with a duty cycle of 38.5%, a frequency of 300 Hz, and potentials ranging from +250 to −24 V. In SEM analysis, the coatings exhibited characteristic PEO treatment morphologies, such as porous volcanic morphology and cracks along the coating, resulting from the rapid cooling of the surface in contact with the aqueous solution. Additionally, they displayed a brownish color, attributed to the high phosphate concentration in the electrolyte, as shown in Figure 6.

Figure 6. SEM micrograph of Ti-6Al-4V alloy treated by PEO (Reproduced from Ref. ^[75] with permission).

DownLoad: Full-Size Img PowerPoint

These findings are corroborated by other studies in the literature. For example, Grigoriev et al. ^[76] observed similar morphological characteristics in oxide coatings produced by PEO on the Ti-6Al-4V alloy, where the porosity and cracks were attributed to the same rapid cooling mechanisms. Additionally, they found that the addition of GO decreases the coefficient of friction due to the formation of a lubricating layer.

The coatings exhibited an average thickness ranging from 2.05 to 23.83 µm, with an inner layer varying from 0.41 to 1.16 µm, which is responsible for the increased wear resistance, as reported by other authors ^[9,14,15,62].

Santos et al. ^[75] emphasized that the produced coatings exhibit a significant increase in hardness, ranging between 400 and 600 HV, compared to the 332 HV (uncoated substrate). This increase correlates with the improved treated material wear, which showed a loss rate of 2.19 × 10⁻⁶ mm³/s compared to the untreated substrate with 9.79 × 10⁻⁵ mm³/s (an improvement of 99.79%).

In the literature, explanations for the formation of micropores attribute it to the development of sparks (micro-arcs) that occur randomly across the sample surface due to the strong electric field. This phenomenon tends to generate very high, localized temperatures (10000–25000 K), causing alumina liquefaction and gas production. These pores result from the release of these gases, according to the illustration in Figure 7 ^[9,11,48,77].

Figure 7. Growth mechanism of PEO coating: (a) beginning of the process before the plasma; (b) beginning of the growth of the oxide coating; and (c) growth of the pores by the plasma.

DownLoad: Full-Size Img PowerPoint

Initially, due to the passivation characteristic of the substrate, a thin dense barrier layer forms on the substrate within the electrolyte solution (Figure 7a). As the applied potential increases, the barrier layer ruptures due to strong electric fields, resulting in the presence of random arcs on the surface, generally around 100 to 120 V, depending on the system configuration (Figure 7b). As the treatment time extends, the density of the electric arcs increases, raising the temperature of the solution and favoring the oxidation reaction. Literature shows that the oxide coating grows not only above the substrate but also penetrates it, allowing for better anchoring of the coating to the substrate (Figure 7c) ^[73,74,77].

In essence, the breakdown mechanism of the passive oxide layer involves three sequential stages: (Ⅰ) first, the rupture of the passive coating occurs due to a sudden rise in temperature from the Joule effect; (Ⅱ) second, the localized aluminum melts from the intense electric discharges' high temperatures (ranging from 10000 to 25000 K), leading to its oxidation; and (Ⅲ) finally, following the cessation of discharges, the aluminum oxide cools upon contact with the liquid, subsequently being deposited on the channels' walls (pores), thereby augmenting the coating's thickness. This process is commonly referred to as "breakdown-melt-ejection-deposition" ^[71,72,78].

In Figure 8, the progression of voltage with respect to time in a direct current system is illustrated. This illustration may be supplemented by Figure 7, wherein Figure 7a is associated with phase I. During this phase, the PEO procedure resembles traditional anodization, characterized by a swift upsurge in voltage and the emission of gas at the anode. Phase 2 is marked by the initiation of sparking (occurring between 120 and 140 V, depending on the specific parameters), along with the formation of pores and the growth of an oxide coating, demonstrated in Figure 7b. In phase 3, the voltage escalation is less abrupt, yet it still facilitates the uninterrupted advancement of the process, as delineated in Figure 7c.

Figure 8. Voltage evolution over time in the PEO process.

DownLoad: Full-Size Img PowerPoint

Hussein, Nie and Northwood ^[79] delineated two distinct growth mechanisms within the framework of the PEO process utilized for the treatment of magnesium alloys; the first is external growth, which entails the expulsion and oxidation of magnesium metallic ions, and the second is internal growth, propelled by the diffusion of oxygen and the depletion of metal within the substrate, as shown in the diagram in Figure 7. These mechanisms give rise to a proportional augmentation in coating thickness in compliance with Faraday's principles of electrolysis. Sudararajan and Krishna ^[80], Matykina et al. ^[81], and Rogov et al. ^[82] also deliberated upon and observed these mechanisms, underscoring the significance of comprehending both external and internal processes involved in the development of PEO coatings on magnesium alloys. This comprehension serves as a pivotal element in the optimization of coating characteristics and the reinforcement of corrosion resistance in applications involving magnesium alloys.

5. Technological and economic comparison between PEO and conventional anodizing

The PEO process and traditional anodizing techniques are widely used to improve the surface characteristics of various light alloys. Table 8 describes a juxtaposition of the main technological attributes and expenses associated with each methodology.

Table 8. Comparison of formation mechanisms and costs of PEO and conventional anodization ^[83,84].

Characteristics	Anodizing	PEO
Formation mechanism	Controlled oxidation in acidic solution with electric current, forming a uniform oxide layer	Electric discharges in an electrolyte solution, resulting in high local temperature and the formation of thick oxide
Compatibility with aluminum alloys	Primarily purer aluminum alloys	Can treat alloys difficult to anodize, such as Al-Si
Process complexity	Simpler, well-established	More complex and requires precise control of discharge parameters
Equipment cost	Moderate, more common, and accessible equipment	High due to the need for high-voltage sources and precise control
Operational cost	Relatively low, less energy consumption	High due to energy consumption and specialized maintenance
Environmental impact	Can use more environmentally friendly electrolytes, but most are acidic	Can be optimized with eco-friendly electrolytes
Typical applications	Architecture, appliances, and decorative applications	Aerospace, automotive, and biomedical industries

| Show Table

DownLoad: CSV

The initial investment required for the installation of a PEO system is significantly elevated owing to the necessity of high-voltage sources and sophisticated control mechanisms, leading to escalated operational expenses from increased electricity consumption. Moreover, the maintenance of equipment is costlier due to the intricate nature of the system. Nevertheless, notwithstanding the considerable expenses incurred, PEO may prove to be a more cost-effective solution in the long run for scenarios where longevity and resistance to corrosion are of paramount importance, thus diminishing the frequency of replacements and repairs ^[83,84].

6. Conclusions

This study contributes to the existing literature with new articles published in the field of surface treatments by conventional anodization and the PEO process. It provides a detailed comparison of the physical and chemical characteristics of the generated coatings, emphasizing the improvements brought by the PEO process in terms of both product performance and environmental impact. Additionally, the PEO process offers significant environmental advantages, including reduced use of toxic chemicals and lower pollutant emissions. This comprehensive analysis reinforces the relevance of PEO as an advanced and sustainable technique for metal surface treatment. The PEO treatment has proven to be of great value both in the academic and industrial sectors for producing oxide coatings on surfaces of lightweight alloys (Al, Ti, Mg, etc.) with optimized properties.

PEO coatings demonstrate comparable or superior results to those obtained by anodizing or hard anodizing (H/A) in basic studies. Examples include research on the corrosion of AA5052 alloys, highlighting a significant improvement of approximately 87% provided by plasma treatment. This is due to the morphology of the PEO coating, which tends to develop a thicker layer and, depending on the system configuration, a denser coating, preventing the material from coming into contact with aggressive environments.

In research focused on joining materials by bonding, the PEO coating can ensure better mechanical anchoring between the joined parts due to the formation of pores, allowing the adhesive to penetrate and solidify, thus ensuring adhesion. Compared to conventional anodizing processes, the PEO process provides the same resistance when subjected to lap shear tests. These results reinforce the superiority of the coatings' characteristics produced by the PEO process, such as the greater thickness, which is attributed to the breakdown of the alumina dielectric layer, promoting more efficient growth.

Many recent works have applied statistical studies to optimize the properties of oxide coatings produced by the PEO process, reducing electricity consumption, which is still one of its weak points, making it not so attractive for low-scale productions. However, when it comes to high-value-added products, PEO applications include treating the surface of turbomolecular pump rotors due to increased abrasion resistance and structures for heat dissipation in various industries such as defense, electronics, automotive, general tools, and nuclear industries. Therefore, future research involving optimizations in energy consumption and high reproducibility will be highly necessary, both in the industrial sector, aiming to reduce costs and, consequently, the price of products, and in the academic sector.

Perspectives for future research

Future research should be directed toward the development and experimental validation of optimized parameters for both the PEO and traditional anodization methods. The advancement of the PEO process, in conjunction with meticulous experimental methodologies, will play a pivotal role in enhancing coating properties, catering to industrial needs, and minimizing energy consumption. Regarding conventional anodization techniques, investigations into enhanced and eco-friendly electrolytes will emerge as notable contributions in compliance with recent regulations and environmental campaigns.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

The authors would like to thank the Coordination for the Improvement of Higher Education Personnel (CAPES) for the financial support provided under grant numbers 88887.827403/2023-00 and 88881.933644/2024-1 (PhD Scholarship–Rafael Resende Lucas).

Author contributions

Writing—original draft: R.R.L., R.C.M.S.C., F.J.G.S., E.C.B. and R.P.M.; writing—review and editing: R.R.L., R.C.M.S.C., F.J.G.S., E.C.B. and R.P.M.

Conflict of interest

Rita De Cássia M. Sales-Contini and Francisco J. G. Silva are on a special issue editorial board for AIMS Materials Science and were not involved in the editorial review or the decision to publish this article. All authors declare that there are no competing interests.

References

[1]	Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature, 521 (2015), 436–444. doi: 10.1038/nature14539. doi: 10.1038/nature14539
[2]	Google AI Blog, AlphaGo: Mastering the ancient game of Go with Machine Learning, 2016. Available from: https://ai.googleblog.com/2016/01/alphago-mastering-ancient-game-of-go.html.
[3]	The 3rd Generation Partnership Project (3GPP). Available from: https://www.3gpp.org/specifications/specifications.
[4]	H. Yang, A. Alphones, Z. Xiong, D. Niyato, J. Zhao, K. Wu, Artificial-intelligence-enabled intelligent 6G networks, IEEE Network, 34 (2020), 272–280. doi: 10.1109/MNET.011.2000195. doi: 10.1109/MNET.011.2000195
[5]	A. Goldsmith, S. Chua, Adaptive coded modulation for fading channels, IEEE Trans. Commun., 46 (1998), 595–602. doi: 10.1109/26.668727. doi: 10.1109/26.668727
[6]	D. Gesbert, S. Kiani, A. Gjendemsjo, G. Oien, Adaptation, coordination, and distributed resource allocation in interference-limited wireless networks, Proc. IEEE, 95 (2007), 2393–2409. doi: 10.1109/JPROC.2007.907125. doi: 10.1109/JPROC.2007.907125
[7]	J. Lu, T. T. Tjhung, F. Adachi, C. L. Huang, BER performance of OFDM-MDPSK system in frequency-selective Rician fading with diversity reception, IEEE Trans. Veh. Technol., 49 (2000), 1216–1225. doi: 10.1109/25.875231. doi: 10.1109/25.875231
[8]	S. T. Chung, A. J. Goldsmith, Degrees of freedom in adaptive modulation: a unified view, IEEE Trans. Commun., 49 (2001), 1561–1571. doi: 10.1109/VETECS.2001.944588. doi: 10.1109/VETECS.2001.944588
[9]	D. Qiao, S. Choi, K. G. Shin, Goodput analysis and link adaptation for IEEE 802.11a wireless LANs, IEEE Trans. Mobile Comput., 1 (2002), 278–291. doi: 10.1109/TMC.2002.1175541 doi: 10.1109/TMC.2002.1175541
[10]	E. Peh, Y. Liang, Y. Guan, Y. Zeng, Optimization of cooperative sensing in cognitive radio networks: A sensing-throughput tradeoff view, IEEE Trans. Veh. Technol., 58 (2009), 5294–5299. doi: 10.1109/TVT.2009.2028030. doi: 10.1109/TVT.2009.2028030
[11]	G. Bianchi, Performance analysis of the IEEE 802.11 distributed coordination function, IEEE J. Sel. Areas Commun., 18 (2000), 535–547. doi: 10.1109/49.840210. doi: 10.1109/49.840210
[12]	J. Mitola, G. Q. Maguire, Cognitive radio: making software radios more personal, IEEE Pers. Commun., 6 (1999), 13–18. doi: 10.1109/98.788210. doi: 10.1109/98.788210
[13]	S. Haykin, Cognitive radio: Brain-empowered wireless communications, IEEE J. Sel. Areas Commun., 23 (2005), 201–220. doi: 10.1109/JSAC.2004.839380. doi: 10.1109/JSAC.2004.839380
[14]	M. G. Kibria, K. Nguyen, G. P. Villardi, O. Zhao, K. Ishizu, F. Kojima, Big data analytics, machine learning, and artificial intelligence in next-generation wireless networks, IEEE Access, 6 (2018), 32328–32338. doi: 10.1109/ACCESS.2018.2837692. doi: 10.1109/ACCESS.2018.2837692
[15]	M. Chen, U. Challita, W. Saad, C. Yin, M. Debbah, Artificial neural networks-based machine learning for wireless networks: A tutorial, IEEE Commun. Surv. Tutor., 21 (2019), 3039–3071. doi: 10.1109/COMST.2019.2926625. doi: 10.1109/COMST.2019.2926625
[16]	J. Wang, C. Jiang, H. Zhang, Y. Ren, K. -C. Chen, L. Hanzo, Thirty years of machine learning: The road to Pareto-optimal wireless networks, IEEE Commun. Surv. Tutor., 22 (2020), 1472–1514. doi: 10.1109/COMST.2020.2965856. doi: 10.1109/COMST.2020.2965856
[17]	M. Kulin, T. Kazaz, I. Moerman, E. De Poorter, End-to-end learning from spectrum data: A deep learning approach for wireless signal identification in spectrum monitoring applications, IEEE Access, 6 (2018), 18484–18501. doi: 10.1109/ACCESS.2018.2818794. doi: 10.1109/ACCESS.2018.2818794
[18]	C. Jiang, H. Zhang, Y. Ren, Z. Han, K. Chen, L. Hanzo, Machine learning paradigms for next-generation wireless networks, IEEE Wireless Commun., 24 (2017), 98–105. doi: 10.1109/MWC.2016.1500356WC. doi: 10.1109/MWC.2016.1500356WC
[19]	R. Li, Z. Zhao, X. Zhou, G. Ding, Y. Chen, Z. Wang, et al., Intelligent 5G: When cellular networks meet artificial intelligence, IEEE Wireless Commun., 24 (2017), 175–183. doi: 10.1109/MWC.2017.1600304WC. doi: 10.1109/MWC.2017.1600304WC
[20]	T. M. Cover, J. A. Thomas, Elements of Information Theory, Wiley, 1991.
[21]	Z. M. Fadlullah, F. Tang, B. Mao, N. Kato, O. Akashi, T. Inoue, et al., State-of-the-art deep learning: Evolving machine intelligence toward tomorrow's intelligent network traffic control systems, IEEE Commun. Surv. Tutor., 19 (2017), 2432–2455. doi: 10.1109/COMST.2017.2707140. doi: 10.1109/COMST.2017.2707140
[22]	H. Ye, G. Y. Li, B. Juang, Power of deep learning for channel estimation and signal detection in OFDM systems, IEEE Wireless Commun. Lett., 7 (2018), 114–117. doi: 10.1109/LWC.2017.2757490. doi: 10.1109/LWC.2017.2757490
[23]	H. Huang, J. Yang, H. Huang, Y. Song, G. Gui, Deep learning for super-resolution channel estimation and DOA estimation based massive MIMO system, IEEE Trans. Veh. Technol., 67 (2018), 8549–8560. doi: 10.1109/TVT.2018.2851783. doi: 10.1109/TVT.2018.2851783
[24]	R. W. Ouyang, A. K. Wong, C. Lea, M. Chiang, Indoor location estimation with reduced calibration exploiting unlabeled data via hybrid generative/discriminative learning, IEEE Trans. Mob. Comput., 11 (2012), 1613–1626. doi: 10.1109/TMC.2011.193. doi: 10.1109/TMC.2011.193
[25]	Y. Chen, Q. Yang, J. Yin, X. Chai, Power-efficient access-point selection for indoor location estimation, IEEE Trans. Knowl. Data. Eng., 18 (2006), 877–888. doi: 10.1109/TKDE.2006.112. doi: 10.1109/TKDE.2006.112
[26]	S. Marano, W. M. Gifford, H. Wymeersch, M. Z. Win, NLOS identification and mitigation for localization based on UWB experimental data, IEEE J. Sel. Areas Commun., 28 (2010), 1026–1035. doi: 10.1109/JSAC.2010.100907. doi: 10.1109/JSAC.2010.100907
[27]	B. Mager, P. Lundrigan, N. Patwari, Fingerprint-based device-free localization performance in changing environments, IEEE J. Sel. Areas Commun., 33 (2015), 2429–2438. doi: 10.1109/JSAC.2015.2430515. doi: 10.1109/JSAC.2015.2430515
[28]	L. U. Khan, I. Yaqoob, M. Imran, Z. Han, C. S. Hong, 6G wireless systems: A vision, architectural elements, and future directions, IEEE Access, 8 (2020), 147029–147044. doi: 10.1109/ACCESS.2020.3015289. doi: 10.1109/ACCESS.2020.3015289
[29]	Y. He, C. Liang, F. R. Yu, Z. Han, Trust-based social networks with computing, caching and communications: A deep reinforcement learning approach, IEEE Trans. Network Sci. Eng., 7 (2020), 66–79. doi: 10.1109/TNSE.2018.2865183. doi: 10.1109/TNSE.2018.2865183
[30]	L. Li, Y. Xu, J. Yin, W. Liang, X. Li, W. Chen, Z. Han, Deep reinforcement learning approaches for content caching in cache-enabled D2D networks, IEEE Internet Things J., 7 (2020), 544–557. doi: 10.1109/JIOT.2019.2951509. doi: 10.1109/JIOT.2019.2951509
[31]	F. B. Mismar, B. L. Evans, A. Alkhateeb, Deep reinforcement learning for 5G networks: Joint beamforming, power control, and interference coordination, IEEE Trans. Commun., 68 (2020), 1581–1592. doi: 10.1109/TCOMM.2019.2961332. doi: 10.1109/TCOMM.2019.2961332
[32]	X. Fu, F. R. Yu, J. Wang, Q. Qi, J. Liao, Dynamic service function chain embedding for NFV-enabled IoT: A deep reinforcement learning approach, IEEE Trans. Wireless Commun., 19 (2020), 507–519. doi: 10.1109/TWC.2019.2946797. doi: 10.1109/TWC.2019.2946797
[33]	H. Huang, Y. Yang, H. Wang, Z. Ding, H. Sari, F. Adachi, Deep reinforcement learning for UAV navigation through massive MIMO technique, IEEE trans. Veh. Technol., 69 (2020), 1117–1121. doi: 10.1109/TVT.2019.2952549. doi: 10.1109/TVT.2019.2952549
[34]	Z. Zhang, H. Chen, M. Hua, C. Li, Y. Huang, L. Yang, Double coded caching in ultra dense networks: Caching and multicast scheduling via deep reinforcement learning, IEEE Trans. Commun., 68 (2020), 1071–1086. doi: 10.1109/TCOMM.2019.2955490. doi: 10.1109/TCOMM.2019.2955490
[35]	H. Robbins, Some aspects of the sequential design of experiments, Bull. Amer. Math. Soc., 58 (1952), 527–535.
[36]	L. Lai, H. E. Jiang, H. V. Poor, Medium access in cognitive radio networks: A competitive multi-armed bandit frame work, in Processing of IEEE 42th Asilomar Conference on Signals, System, and Computer (2008), 98–102. doi: 10.1109/ACSSC.2008.5074370.
[37]	L. Lai, H. E. Jiang, H. V. Poor, Cognitive medium access: Exploration, exploitation, and competition, IEEE Trans. Mobile Comput., 10 (2011), 239–253. doi: 10.1109/TMC.2010.65. doi: 10.1109/TMC.2010.65
[38]	S. Maghsudi, E. Hossain, Multi-armed bandits with application to 5G small cells, IEEE Wireless Commun., 23 (2016), 64–73. doi: 10.1109/MWC.2016.7498076. doi: 10.1109/MWC.2016.7498076
[39]	S. J. Kim, M. Aono, M. Hara, Tug-of-war model for the two-bandit problem: Nonlocally-correlated parallel exploration via resource conservation, Biosystems, 101 (2010), 29–36. doi: 10.1016/j.biosystems.2010.04.002. doi: 10.1016/j.biosystems.2010.04.002
[40]	Z. Zhao, E. Schiller, E. Kalogeiton, T. Braun, B. Stiller, M. T. Garip, et al., Autonomic communications in software-driven networks, IEEE J. Sel. Areas Commun., 35 (2017), 2431–2445. doi: 10.1109/JSAC.2017.2760354. doi: 10.1109/JSAC.2017.2760354
[41]	Radio Spectrum Policy Group, Report on Collective Use of Spectrum and Other Sharing Approaches, 2011. Available from: https://rspg-spectrum.eu/wp-content/uploads/2013/05/rspg11_392_report_CUS_other_approaches_final.pdf.
[42]	CEPT Electronic Communications Committee, ECC Report 205–Licensed Shared Access (LSA), 2014. Available from: http://spectrum.welter.fr/international/cept/ecc-reports/ecc-report-205-LSA-2300-MHz-2400-MHz.pdf.
[43]	Reconfigurable Radio Systems (RRS), Information Elements and Protocols for the Interface between the LSA Controller (LC) and LSA Repository (LR) for the Operation of Licensed Shared Access (LSA) in the 2300 MHz–2400 MHz Band, 2017. Available from: https://www.etsi.org/deliver/etsi_ts/103300_103399/103379/01.01.01_60/ts_103379v010101p.pdf
[44]	S. Haykin, P. Setoodeh, S. Feng, D. Findlay, Cognitive dynamic system as the brain of complex networks, IEEE J. Sel. Areas Commun., 34 (2016), 2791–2800. doi: 10.1109/JSAC.2016.2605240. doi: 10.1109/JSAC.2016.2605240
[45]	M. Hasegawa, H. Hirai, K. Nagano, H. Harada, K. Aihara, Optimization for centralized and decentralized cognitive radio networks, Proc. IEEE, 102 (2014), 574–584. doi: 10.1109/JPROC.2014.2306255. doi: 10.1109/JPROC.2014.2306255
[46]	Y. Kon, K. Hashiguchi, M. Ito, M. Hasegawa, K. Ishizu, H. Murakami, et al., Autonomous throughput improvement scheme using machine learning algorithms for heterogeneous wireless networks aggregation, IEICE Trans. Commun., 95 (2012), 1143–1151. doi: 10.1587/transcom.E95.B.1143. doi: 10.1587/transcom.E95.B.1143
[47]	IEEE, IEEE Standard for Architectural Building Blocks Enabling Network-Device Distributed Decision Making for Optimized Radio Resource Usage in Heterogeneous Wireless Access Networks, 2009. Available from: https://ieeexplore.ieee.org/document/4798288.
[48]	R. Combes, A. Proutiere, Dynamic rate and channel selection in cognitive radio systems, IEEE J. Sel. Areas Commun., 33 (2015), 910–921. doi: 10.1109/JSAC.2014.2361084. doi: 10.1109/JSAC.2014.2361084
[49]	D. Shiung, Y. Yang, Rate enhancement for cognitive radios using the relationship between transmission rate and signal-to-interference ratio statistics, IET Commun., 7 (2013). doi: 10.1049/iet-com.2013.0047. doi: 10.1049/iet-com.2013.0047
[50]	J. Lehtomaki, M. Benitez, K. Umebayashi, M. Juntti, Improved channel occupancy rate estimation, IEEE Trans. Commun., 63 (2015), 643–654. doi: 10.1109/TCOMM.2015.2402195. doi: 10.1109/TCOMM.2015.2402195
[51]	B. E. Boser, I. M. Guyon, V. N. Vapnik, A training algorithm for optimal margin classifiers, in Proceedings of the fifth annual workshop on Computational learning theory, (1992), 144–152. doi: 10.1145/130385.130401.
[52]	V. Vapnik, S. Golowich, A. Smola, Support vector method for function approximation, regression estimation, and signal processing, Adv. Neural Inf. Process. Syst., 1997 (1997), 281–287.
[53]	K. Oshima, T. Kobayashi, Y. Taenaka, K. Kuroda, M. Hasegawa, Wireless network optimization method based on cognitive cycle using machine learning, IEICE Commun. Express, 7 (2018), 278–283. doi: 10.1587/comex.2018XBL0061. doi: 10.1587/comex.2018XBL0061
[54]	K. Oshima, T. Kobayashi, Y. Taenaka, K. Kuroda, M. Hasegawa, Autonomous wireless system optimization method based on cross-layer modeling using machine learning, in IEEE International Conference on Ubiquitous and Future Networks, (2019), 239–244. doi: 10.1109/ICUFN.2019.8806031.
[55]	J. Kennedy, R. Eberhart, Particle swarm optimization, in Proceedings of ICNN'95-International Conference on Neural Networks, 4 (1995), 1942–1948. doi: 10.1109/ICNN.1995.488968.
[56]	V. Kadirkamanathan, K. Selvarajah, P. J. Fleming, Stability analysis of the particle dynamics in particle swarm optimizer, IEEE Trans. Evol. Comput., 10 (2006), 245–255. doi: 10.1109/TEVC.2005.857077. doi: 10.1109/TEVC.2005.857077
[57]	Scalable Network Technologies, Inc., Available from: https://www.scalable-networks.com.
[58]	Beyond 5G/6G White Paper, National Institute of Information and Communications Technology, 2021. Available from: https://beyond5g.nict.go.jp/en/download/index.html.
[59]	P. V. R. Ferreira, R. Paffenroth, A. M. Wyglinski, T. M. Hackett, S. G. Bilen, R. C. Reinhart, et al., Multiobjective reinforcement learning for cognitive satellite communications using deep neural network ensembles, IEEE J. Sel. Areas Commun., 36 (2018). doi: 10.1109/JSAC.2018.2832820. doi: 10.1109/JSAC.2018.2832820
[60]	N. Cardwell, Y. Cheng, C. S. Gunn, S. H. Yeganeh, V. Jacobson, BBR: congestion-based congestion control, ACM Queue, 14 (2016), 20–53.
[61]	H. Yang, Z. Xiong, J. Zhao, D. Niyato, Q. Wu, L. Xiao, Deep reinforcement learning-based intelligent reflecting surface for secure wireless communications, IEEE Trans. Wireless Commun., 20 (2021), 375–388. doi: 10.1109/TWC.2020.3024860. doi: 10.1109/TWC.2020.3024860
[62]	K. Oshima, T. Onishi, S. J. Kim, J. Ma, M. Hasegawa, Efficient wireless network selection by using multi-armed bandit algorithm for mobile terminals, Nonlinear Theory Its Appl. IEICE, 11 (2020), 68–77. doi: 10.1587/nolta.11.68. doi: 10.1587/nolta.11.68
[63]	R. Sutton, A. Barto, Reinforcement Learning: An Introduction, The MIT Press, 1998.
[64]	N. D. Daw, J. P. O'doherty, P. Dayan, B. Seymour, R. J. Dolan, Cortical substrates for exploratory decisions in humans, Nature, 441 (2006), 876–879. doi: 10.1038/nature04766. doi: 10.1038/nature04766
[65]	P. Auer, N. Cesa-Bianchi, P. Fischer, Finite-time analysis of the multiarmed bandit problem, Mach. Learn., 47 (2002), 235–256. doi: 10.1023/A:1013689704352. doi: 10.1023/A:1013689704352
[66]	S. J. Kim, M. Aono, Amoeba-inspired algorithm for cognitive medium access, NOLTA, 5 (2014), 198–209. doi: 10.1587/nolta.5.198. doi: 10.1587/nolta.5.198
[67]	S. J. Kim, M. Aono, E. Nameda, Efficient decision-making by volume-conserving physical object, New J. Phys., 17 (2015). doi: 10.1088/1367-2630/17/8/083023. doi: 10.1088/1367-2630/17/8/083023
[68]	S. J. Kim, M. Aono, E. Nameda, Decision maker based on atomic switches, AIMS Mater. Sci., 3 (2016), 245–259. doi: 10.3934/matersci.2016.1.245. doi: 10.3934/matersci.2016.1.245
[69]	J. Ma, S. Hasegawa, S. J. Kim, M. Hasegawa, A reinforcement-learning-based distributed resource selection algorithm for massive IoT, Appl. Sci., 9 (2019). doi: 10.3390/app9183730. doi: 10.3390/app9183730
[70]	H. Yang, Z. Xiong, J. Zhao, D. Niyato, Q. Wu, H. V. Poor, et al., Intelligent reflecting surface assisted anti-jamming communications: A fast reinforcement learning approach, IEEE Trans. Wireless Commun., 20 (2021), 1963–1974. doi: 10.1109/TWC.2020.3037767. doi: 10.1109/TWC.2020.3037767
[71]	S. Singh, J. G. Andrews, Joint resource partitioning and offloading in heterogeneous cellular networks, IEEE Trans. Wireless Commun., 13 (2014), 888–901. doi: 10.1109/TWC.2013.120713.130548. doi: 10.1109/TWC.2013.120713.130548
[72]	C. Liu, M. Li, S. V. Hanly, P. Whiting, Joint downlink user association and interference management in two-tier HetNets with dynamic resource partitioning, IEEE Trans. Veh. Technol., 66 (2017), 1365–1378. doi: 10.1109/TVT.2016.2565538. doi: 10.1109/TVT.2016.2565538
[73]	V. Sagar, R. Chandramouli, K. P. Subbalakshmi, Software defined access for HetNets, IEEE Commun. Mag., 54 (2016), 84–89. doi: 10.1109/MCOM.2016.7378430. doi: 10.1109/MCOM.2016.7378430
[74]	A. Keshavarz-Haddad, E. Aryafar, M. Wang, M. Chiang, HetNets selection by clients: Convergence efficiency and practicality, IEEE/ACM Trans. Netw., 25 (2017), 406–419. doi: 10.1109/TNET.2016.2587622. doi: 10.1109/TNET.2016.2587622
[75]	E. Aryafar, A. Keshavarz-Haddad, M. Wang, M. Chiang, RAT selection games in HetNets, in 2013 Proceedings IEEE INFOCOM, (2013), 998–1006. doi: 10.1109/INFCOM.2013.6566889.
[76]	X. Wang, J. Li, L. Wang, C. Yang, Z. Han, Intelligent user-centric network selection: A model-driven reinforcement learning framework, IEEE Access, 7 (2019), 21645–21661. doi: 10.1109/ACCESS.2019.2898205. doi: 10.1109/ACCESS.2019.2898205
[77]	D. D. Nguyen, H. X. Nguyen, L. B. White, Reinforcement learning with network-assisted feedback for heterogeneous RAT selection, IEEE Trans. Wireless Commun., 19 (2017). doi: 10.1109/TWC.2017.2718526. doi: 10.1109/TWC.2017.2718526
[78]	K. Kuroda, H. Kato, S. J. Kim, M. Naruse, M. Hasegawa, Improving throughput using multi-armed bandit algorithm for wireless LANs, NOLTA, 9 (2018), 74–81. doi: 10.1587/nolta.9.74. doi: 10.1587/nolta.9.74
[79]	ns-3, ns-3: A Discrete-event Network Simulator for Internet Systems, Available from: https://www.nsnam.org/.
[80]	S. Takeuchi, M. Hasegawa, K. Kanno, A. Uchida, N. Chauvet, M. Naruse, Dynamic channel selection in wireless communications via a multi-armed bandit algorithm using laser chaos time series, Sci. Rep., 10 (2020). doi: 10.1038/s41598-020-58541-2. doi: 10.1038/s41598-020-58541-2
[81]	H. Kanemasa, A. Li, M. Naruse, N. Chauvet, M. Hasegawa, Dynamic channel bonding using laser chaos decision maker in WLANs, 2021 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), (2021), 078–082. doi: 10.1109/ICAIIC51459.2021.9415227.
[82]	Z. Duan, N. Okada, A. Li, M. Naruse, N. Chauvet, M. Hasegawa, High-speed optimization of user pairing in NOMA system using laser chaos based MAB algorithm, 2021 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), (2021), 073–077. doi: 10.1109/ICAIIC51459.2021.9415234.

This article has been cited by:

Salvacion B. Orgen, Eden May B. Dela Pena, Microstructure and Corrosion Behavior of PEO-Coated AA7075 Under Pulsed Unipolar Potential Control Mode, 2024, 14, 2079-6412, 1498, 10.3390/coatings14121498

Reader Comments

Your name:*

Email:*
© 2022 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)