
Deep reinforcement learning algorithms are widely used in the field of robot control. Sparse reward signals lead to blind exploration, affecting the efficiency of the manipulator during path planning for multi-axis systems at any given end-effector start and target position. To address the problem of tracking randomly located targets in three-dimensional space, this paper proposes a PPO (proximal policy optimization) algorithm with a fused reward mechanism, which enhances the tracking and guidance capabilities of the manipulator in multiple dimensions and reduces the blind randomness of the manipulator during the detection and sampling process. The fusion reward mechanism consists of four dimensions: trajectory correction reward, core area acceleration guidance reward, ladder adaptability reward, and abnormal termination penalty. Finally, a 7-degree-of-freedom Kuka manipulator is built on the PyBullet platform for simulation experiments. Experimental results show that, compared with the sparse reward mechanism, the PPO algorithm with the fused reward mechanism has a higher average success rate as high as 94.88% in task tracking, which can effectively improve the tracking efficiency and accuracy of the spatial manipulator.
Citation: Ruyi Dong, Kai Yang, Tong Wang. Research on tracking strategy of manipulator based on fusion reward mechanism[J]. AIMS Electronics and Electrical Engineering, 2025, 9(1): 99-117. doi: 10.3934/electreng.2025006
[1] | Supriya Rattan, Derek Fawcett, Gerrard Eddy Jai Poinern . Williamson-Hall based X-ray peak profile evaluation and nano-structural characterization of rod-shaped hydroxyapatite powder for potential dental restorative procedures. AIMS Materials Science, 2021, 8(3): 359-372. doi: 10.3934/matersci.2021023 |
[2] | Saviska Luqyana Fadia, Istie Rahayu, Deded Sarip Nawawi, Rohmat Ismail, Esti Prihatini, Gilang Dwi Laksono, Irma Wahyuningtyas . Magnetic characteristics of sengon wood-impregnated magnetite nanoparticles synthesized by the co-precipitation method. AIMS Materials Science, 2024, 11(1): 1-27. doi: 10.3934/matersci.2024001 |
[3] | Mohammed Tareque Chowdhury, Md. Abdullah Zubair, Hiroaki Takeda, Kazi Md. Amjad Hussain, Md. Fakhrul Islam . Optical and structural characterization of ZnSe thin film fabricated by thermal vapour deposition technique. AIMS Materials Science, 2017, 4(5): 1095-1121. doi: 10.3934/matersci.2017.5.1095 |
[4] | Omar Hassan Mahmood, Mustafa Sh. Aljanabi, Farouk M. Mahdi . Effect of Cu nanoparticles on microhardness and physical properties of aluminum matrix composite prepared by PM. AIMS Materials Science, 2025, 12(2): 245-257. doi: 10.3934/matersci.2025013 |
[5] | Raghad Subhi Abbas Al-Khafaji, Kareem Ali Jasim . Dependence the microstructure specifications of earth metal lanthanum La substituted Bi2Ba2CaCu2-XLaXO8+δ on cation vacancies. AIMS Materials Science, 2021, 8(4): 550-559. doi: 10.3934/matersci.2021034 |
[6] | Saif S. Irhayyim, Hashim Sh. Hammood, Anmar D. Mahdi . Mechanical and wear properties of hybrid aluminum matrix composite reinforced with graphite and nano MgO particles prepared by powder metallurgy technique. AIMS Materials Science, 2020, 7(1): 103-115. doi: 10.3934/matersci.2020.1.103 |
[7] | Khang Duy Vu Nguyen, Khoa Dang Nguyen Vo . Magnetite nanoparticles-TiO2 nanoparticles-graphene oxide nanocomposite: Synthesis, characterization and photocatalytic degradation for Rhodamine-B dye. AIMS Materials Science, 2020, 7(3): 288-301. doi: 10.3934/matersci.2020.3.288 |
[8] | Andrew H. Morgenstern, Thomas M. Calascione, Nathan A. Fischer, Thomas J. Lee, John E. Wentz, Brittany B. Nelson-Cheeseman . Thermoplastic magnetic elastomer for fused filament fabrication. AIMS Materials Science, 2019, 6(3): 363-376. doi: 10.3934/matersci.2019.3.363 |
[9] | Purabi R. Ghosh, Derek Fawcett, Michael Platten, Shashi B. Sharma, John Fosu-Nyarko, Gerrard E. J. Poinern . Sustainable green chemical synthesis of discrete, well-dispersed silver nanoparticles with bacteriostatic properties from carrot extracts aided by polyvinylpyrrolidone. AIMS Materials Science, 2020, 7(3): 269-287. doi: 10.3934/matersci.2020.3.269 |
[10] | Andrea Somogyi, Cristian Mocuta . Possibilities and Challenges of Scanning Hard X-ray Spectro-microscopy Techniques in Material Sciences. AIMS Materials Science, 2015, 2(2): 122-162. doi: 10.3934/matersci.2015.2.122 |
Deep reinforcement learning algorithms are widely used in the field of robot control. Sparse reward signals lead to blind exploration, affecting the efficiency of the manipulator during path planning for multi-axis systems at any given end-effector start and target position. To address the problem of tracking randomly located targets in three-dimensional space, this paper proposes a PPO (proximal policy optimization) algorithm with a fused reward mechanism, which enhances the tracking and guidance capabilities of the manipulator in multiple dimensions and reduces the blind randomness of the manipulator during the detection and sampling process. The fusion reward mechanism consists of four dimensions: trajectory correction reward, core area acceleration guidance reward, ladder adaptability reward, and abnormal termination penalty. Finally, a 7-degree-of-freedom Kuka manipulator is built on the PyBullet platform for simulation experiments. Experimental results show that, compared with the sparse reward mechanism, the PPO algorithm with the fused reward mechanism has a higher average success rate as high as 94.88% in task tracking, which can effectively improve the tracking efficiency and accuracy of the spatial manipulator.
In recent years, ferroic materials have attracted considerable scientific interest due to their intriguing and unique physical and chemical properties. In particular, magnetite (Fe3O4) exhibits distinctive structural, electronic, and magnetic properties [1]. Bulk scale magnetite displays a cubic-inverse spinel arrangement, while magnetite nanoparticles (MNPs) have a mixed spinel form due to increased surface strains. The mixed spinel form also results in two distinct crystallographic sites, namely tetrahedral and octahedral. The distribution of divalent (Fe2+) and trivalent (Fe3+) cations in the respective tetrahedral and octahedral sites plays an important part in influencing the properties of ferrite materials [2]. In turn, these properties influence the type of function that MNPs can deliver in specific applications. To date, MNPs have been used in a wide variety of applications, including: (1) imaging agents in magnetic resonance (MRI) analysis [3,4,5,6,7]; (2) solid component in ferro-fluids [4]; (3) agents in hyperthermia therapy [3,4,7,8,9]; (4) carrier agents in bio-sensing and diagnosis [7,10,11]; (5) drug delivery platforms for pharmaceuticals [3,4,7,10,11,12,13]; (6) adjuvant and carrier agent in cancer treatments [7,8,9,14,15]; (7) electronics and data storage devices [15,16]; (8) catalyst supports [4,15,17,18]; (9) incorporated into magnetic paints and inks [15,19]; (10) components in energy storage systems [20,21,22,23]; and (11) adsorbents for the removal of contaminants from wastewater [18,24,25,26,27].
The wide variety of applications in which MNPs are utilized is a direct result of their size, composition, purity, and material properties. Importantly, nanoparticle size and its properties can be regulated by the type of fabrication process used in its manufacture. MNPs can be generated by either top-down or bottom-up methods. Top-down methods include several techniques, such as photolithography, ball milling, and grinding [28]. Bottom-up methods include pyrolysis [29], micro-emulsion [30], hydrothermal/solvo-thermal [1,31], sol-gel [32], sonochemical and microwave-assisted synthesis [33], chemical co-precipitation [34,35], and biosynthesis [29]. Furthermore, in recent years, there has been significant scientific interest in developing biological synthesis methods. These methods have the potential to replace several conventional chemical and physical methods and deliver better, greener, and less energy-intensive synthesis procedures [36]. Many of these conventional chemical and physical methods also have disadvantages like high manufacturing costs, the usage and disposal of harmful chemicals, and the creation of harmful by-products [37,38,39]. Thus, using alternative biogenic methods and the principle of green chemistry avoids these disadvantages. Biogenic synthesis has focused mainly on two types of biological entities, namely bacterial and plant-based techniques [39,40]. In recent years, a significant research effort has focused on plant-based techniques to produce nanoparticles. Studies have shown that extracts taken from stems, leaves, fruits, and seeds can be used to synthesize nanoparticles [40,41,42,43]. Nanoparticle synthesis via plant extracts is due to the presence of phytochemicals (e.g., amino acids, polyphenols, or reducing sugars), which act as stabilizing and reducing agents [43].
One bottom-up method for generating MNPs that has attracted significant interest in recent years involves iron salt hydroxylation in alkaline solutions followed by the dehydration of the resulting hydroxides. The method's advantages include: (1) use of readily available precursors; (2) being a straightforward chemical synthesis process; (3) not relying on complex and expensive laboratory equipment; (4) being more eco-friendly when compared to other methods that use hazardous chemicals and solvents, and (5) having the potential to be scaled up to facilitate large scale production [44].
The present exploratory study evaluates the use of a leaf extract taken from the Australian indigenous plant Banksia ashbyi (BA) as an agent for controlling nanoparticle size and size distribution when incorporated into the iron salt hydroxylation/dehydration method mentioned above. In this one-pot, facile, and green method, the aqueous BA leaf extract first assists in reducing the Fe precursors and then acts as a capping agent to form stable MNPs. Advanced characterization techniques were used to investigate the physiochemical and material properties of the MNPs. Fourier transform infrared spectroscopy (FT-IR), energy dispersive spectroscopy (EDS), and Raman spectroscopy confirmed the formation and chemical nature of the MNPs. UV-visible (UV-Vis) spectroscopy was used to determine the band gap, and thermo-gravimetric analysis (TGA) was used to evaluate the thermal stability of the MNPs. The geometric parameters of the MNPs including crystallite size and particle size were studied using X-ray diffraction (XRD), focused ion beam scanning electron microscopy (FIBSEM) and transmission electron microscopy (TEM). In addition, X-ray peak profile analysis using Williamson-Hall methods was used to determine strain, stress, and strain energy due to lattice deformation of the MNPs. XRD analysis was used to determine material properties, such as Young's modulus of elasticity, modulus of rigidity, and Poisson's ratio.
Physical properties, such as crystallite size, lattice stress, lattice strain, and elastic modulus can be estimated from X-ray peak profile analysis using Williamson-Hall analysis. Peak broadening occurs in X-ray diffraction spectra due to instrumental effects and imperfect sample crystallinity. To correct for instrumental broadening and remove this aberration, it is necessary to collect a diffraction pattern using a standard material such as silicon. Then, the peaks in the silicon diffraction pattern are used to correct the corresponding peaks of the sample diffraction pattern using Eq 1 [45]:
β(hkl)=[β2(hkl)(measured)−β2(hkl)(instrumental)]0.5 | (1) |
Then, the crystalline size (D(hkl)) of the sample is calculated using the Debye-Scherer formula presented in Eq 2 [46]:
D(hkl)=kλβ(hkl) cos θ(hkl) | (2) |
In Eq 2, λ is the wavelength of the monochromatic X-ray beam, and k is the crystallite shape constant, which is 0.9 for spherical crystals with cubic unit cells. β(hkl) is the full width at half maximum (FWHM) of the peak at the maximum intensity, and θ(hkl) is the peak diffraction angle that satisfies Bragg's law for the (h k l) plane. D(hkl) is the crystallite size of the sample. Eq 2 also shows that the crystalline size (D(hkl)) is dependent on the reciprocal of cosine θ. The crystallographic distance (d) between planes for given Miller indices h, k, and l can be estimated using Bragg's equation, as presented by Eq 3:
nλ=2d sin θ | (3) |
In Eq 3, n and θ represent the diffraction order (usually n = 1) and the Bragg angle, respectively. In the case of a cubic structure, the d spacing is related to the lattice constants and Miller indices h, k, and l through Eq 4. Thus, Eq 4 can be used to estimate the size of the respective lattice constants.
d=a√h2+k2+l2 | (4) |
Once the lattice constant (a) is determined, then the unit-cell volume can be estimated using Eq 5 [47]:
V=a3 | (5) |
Studies have shown that crystallite size and strain both contribute to the peak broadening and are independent of each other [45,46]. Both parameters can be expressed by Eq 6:
β(hkl)=β(crystallite size)+β(strain) | (6) |
The strain-induced peak broadening component of Eq 6 can be determined from Eq 7, which shows the strain (ε) is inversely proportional to tan (θ):
ε=β(hkl)4 tan θ | (7) |
Thus, combining the crystallite size and strain-induced peak broadening components expressed in Eqs 2 and 7, Eq 8 below can be derived:
β(hkl)=kλD(hkl) cos θ(hkl)+4ε tan θ | (8) |
Then, rearranging Eq 8 gives Eq 9. Both Eqs 8 and 9 form the basis of Williamson-Hall analysis of peak broadening effects seen in sample powders.
β(hkl) cos θ=kλD(hkl)+4ε sin θ | (9) |
Eq 9 represents the uniform deformation model (UDM), which assumes a uniform strain in all crystallographic directions and that material properties are independent of the direction along which measurements are made. When the resulting experimental measurements are plotted, with 4sinθ along the x-axis and β(hkl) cosθ along the y-axis, the resulting graph gives a linear fit to the data. From the graph, strain (slope) and crystallite size (y-intercept) can be determined. The second Williamson-Hall analysis uses the uniform stress deformation model (USDM), which assumes the sample material follows Hooke's law. Thus, there is a linear relationship between the stress and strain, as seen in Hooke's law expressed by Eq 10.
E=σε | (10) |
In Eq 10, E is the elastic modulus, also known as Young's modulus of elasticity, and σ is the stress present in the crystallite structure. Incorporating Eq 10 into Eq 9 gives Eq 11, which is used in the (USDM) analysis.
β(hkl) cos θ=kλD(hkl)+4σ sin θE(hkl) | (11) |
In Eq 11, E(hkl) is the modulus of elasticity occurring perpendicular to the (hkl) crystal lattice plane. Because E(hkl) is dependent on the crystallographic direction, it must be calculated for each orientation within the cubic crystal structure, as seen in Eq 12 [48,49].
1E(hkl)=S11−2(S11−S12−12S44)(l21l22+l22l23+l23l21) | (12) |
In Eq 12, S11, S12, and S44 are the elastic-compliances, and l1, l2, and l3 are the orientation parameters derived from Eqs 13–15, respectively.
l1=h(h2+k2+l2)−0.5 | (13) |
l2=k(h2+k2+l2)−0.5 | (14) |
l3=l(h2+k2+l2)−0.5 | (15) |
In addition, the elastic compliances (S11, S12, and S44) are related to the stiffness parameters (C11, C12, and C44) of the material through Eqs 16–18 below.
S11=(C11+C12)(C11−C12)(C11+2C12) | (16) |
S12=−C12(C11−C12)(C11+2C12) | (17) |
S44=1C44 | (18) |
Once the respective elastic-compliances and orientation parameters are derived, then the E(hkl) for each crystal plane can be calculated. Next, the respective E(hkl) values are then substituted into Eq 11. Then, by plotting 4sinθ/E(hkl) along the x-axis and β(hkl) cosθ along the y-axis, the resulting graph gives a linear fit to the data. From the graph of the USDM data, the slope of the graph is the stress present in the sample and the y-intercept is the estimated crystallite size. However, in many cases, the assumption of homogeneity and isotropy in all crystallographic directions is not achieved. When strain energy density (u) is considered, the proportionality normally associated with the stress and strain relationship may no longer be independent. The strain energy density (u) can be calculated using Eq 19 presented below. When the strain energy density equation is rearranged and substituted into Eq 9, the modified equation (using Eq 20) becomes the uniform deformation energy density model (UDEDM) [50].
u=ε2E(hkl)2 | (19) |
β(hkl) cos θ=kλD(hkl)+4 sin θ(2uE(hkl))12 | (20) |
Thus, by plotting 4sinθ(2/E(hkl))1/2 along the x-axis and β(hkl)cosθ along the y-axis, the resulting slope gives the lattice strain, and the y-intercept gives the crystallite size. Once the lattice strain is determined, the respective strain energy density value can be calculated.
Further analysis of the XRD data can be used to determine physical properties such as: (1) density (ρXRD); (2) longitudinal (VL), shear (VS), and mean (Vm) wave velocities; (3) longitudinal modulus (L); (4) modulus of rigidity (G); (5) bulk modulus (B); and (6) Poisson's ratio of the sample material.
The X-ray density (ρXRD) is calculated using Eq 21, where M = molecular weight (g mol−1 and for magnetite 231.54 g mol−1), NA = 6.022 × 1023 mol−1 (Avogadro's number), z = unit cell number, in this case 8, and a = lattice parameter determined from XRD data.
ρ(XRD)=MNA×za3 | (21) |
The respective longitudinal (VL), shear (VS), and mean (Vm) wave velocities are calculated from Eqs 22–24, respectively. Both stiffness parameters C11 and C44 are determined as discussed in the previous section.
VL=(C11ρ)12 | (22) |
VS=(C44ρ)12=VL√3 | (23) |
3V3m=1V3L+2V3S | (24) |
Once the wave velocity values VL and VS are determined, they can be used to determine other physical properties present in the sample material. The longitudinal modulus (L) can be determined using Eq 25, and the modulus of rigidity (G) can be determined using Eq 26 [51].
L=ρ×(V1)2 | (25) |
G=ρ×(VS)2 | (26) |
The values obtained from Eqs 25 and 26 can then be substituted into Eq 27 to determine Poisson's ratio.
σ=(L−2G)2(L−G) | (27) |
In addition, the bulk modulus (B) of the sample is calculated using Eq 28 and the stiffness parameters C11 and C44 determined in the previous section.
B=13×(C11+2C12) | (28) |
Ferrous sulfate heptahydrate (FeSO4·7H2O), ferric chloride (FeCl3), and aqueous ammonia (NH4OH, 28% w/w.) were purchased from Chem-Supply Pty Ltd, Australia. The analytical-grade ethanol used in the washing stages was supplied by Rowe Scientific Pty Ltd, Australia. All chemicals and solvents were used as supplied by the respective provider. All aqueous-based solutions prepared in this study were prepared using Milli-Q® water (18.3 M Ω cm−1) generated by an ultrapure water system (Barnstead Ultrapure Water System D11931; Thermo Scientific, Dubuque, IA).
A subset of healthy leaves was selected from BA plants located around the Murdoch University campus, Perth, Western Australia (32.0680° S, 115.8352° E). Selected leaves were thoroughly rinsed three times using Milli-Q® water to remove surface contaminants. Cleaned leaves were placed into a pre-heated oven at a temperature of 60 ℃ for 6 h to remove surface moisture. After the drying period, leaves were taken from the oven and individual leaves were cut into square (1 × 1 cm) pieces. From these leaf squares, 2 g was selected and placed into a glass beaker containing 100 mL of Milli-Q® water. This leaf-containing solution was heated to 80 ℃ and then maintained at this temperature for 120 min. During this period, the leaf-containing solution was constantly stirred, and the initial clear aqueous solution slowly changed to a yellow color. After 120 min, heat was removed and the solution containing the leaves and leaf extract was allowed to naturally cool down to room temperature (24 ℃). Following the cooling period, the solution was filtered (Whatman® glass microfiber filters, Grade GF/A) to remove the filtrate and produce the yellow-colored BA leaf extract solution (pH 5). The experimental procedure is schematically presented in Figure 1.
Magnetite nanoparticles were synthesized using a co-precipitation method. Two types of nanoparticles were produced in individual 50 mL test solutions prepared in small flasks. The first type was the control and its starting solution was a 50 mL solution of Milli-Q® water. The magnetite nanoparticles produced in the aqueous solution were designated as MNPs. The second type of magnetite nanoparticles were created in a 50 mL solution of BA extract and these magnetite nanoparticles were designated as BA-MNPs. The test solutions were placed on a hotplate and heated to 80 ℃. Next, FeCl3 and FeSO4·7H2O, in a molar ratio of 2:1, were added to each of the test solutions. The resulting reaction was then allowed to run for 30 min at 80 ℃ while being constantly stirred. During this period, the color of the BA extract solution changed from yellow to brown. At the end of the 30 min period, a 10 mL solution of NH4OH (28% w/w.) was added dropwise to each test solution to increase the solution pH to between 11 and 12. After a further 15 min of stirring at 80 ℃, a sizeable amount of black precipitate accumulated at the bottom of each flask. Then, both heating and stirring were stopped, and an external neodymium magnet was used to collect and remove the black precipitate from each of the test solutions. Before the respective black precipitates were vacuum-dried at 80 ℃ for 6 h, they were washed three times with Milli-Q® water and finally washed with an ethanol solution. The overall reaction of the BA-assisted synthesis of MNPs is presented in Eq 29 below, and the experimental procedure is schematically presented in Figure 1.
BA leaf extract +Fe2++2Fe3++8OH−→(BA−MNPs) capped with phyto-molecules +4H2O | (29) |
X-ray diffraction was used to study the crystalline nature and particle size distribution of the samples using a Rigaku SmartLab X-ray diffractometer. The 2θ values were collected over the range 5–80°, with step angle increments of 0.01° and at a speed of 1°/min. The diffractometer used a Cu-Kα radiation source (λ = 1.5406 Å) and operated at 45 kV and 200 mA. Two electron microscopy studies were conducted to determine particle size, structural features, and morphology using a Zeiss Neon 40EsB FIBSEM located in the John de Laeter Center at Curtin University and a Philips CM200 TEM located at the FEG-TEM facility at Sophisticated Analytical Instrument Facility (SAIF), Indian Institute of Technology, Mumbai (India), for high-resolution images. In addition, the Zeiss Neon also used its energy dispersive spectroscopy (EDS) attachment to determine the compositional analysis of the samples and the Philips CM200 also provided selected area electron diffraction (SAED) patterns of the samples. For the Zeiss Neon, samples were deposited on carbon tape-covered SEM holders and then sputter coated (E5000, Polaron Equipment Ltd.) with a 2 nm thick layer of platinum to prevent charge build-up. Samples for the Philips CM200 TEM were prepared by placing the samples on a carbon-coated TEM support grid (copper) and then sputter-coated with a 10 nm thick layer of gold to prevent charge build-up. A PerkinElmer FT-IR/NIR Spectrometer Frontier fitted with a universal signal bounce Diamond ATR attachment was used to carry out an FT-IR study of the samples to identify the various functional groups present. The analysis was carried out over the wavenumber interval between 400 and 4000 cm−1, with a wavenumber resolution step of 1 cm−1. Raman spectroscopy was used to determine the molecular vibrations and chemical compositions of the various samples. Measurements were made using a LabRAM 1B Raman spectrometer that used a 632.82 nm Helium-Neon laser light source. Analysis was carried out over the wavenumber interval between 0 and 3000 cm−1, with a wavenumber resolution step of 1 cm−1. UV-vis was carried out using a Perkin Elmer STA 8000 spectrometer over a spectral range between 300 and 1100 nm, with a range resolution of 1 nm. TGA was conducted to evaluate the thermal stability of the respective samples using a Perkin Elmer simultaneous thermal analyzer STA 8000. Samples were placed into a ceramic (Alumina) crucible and loaded into the STA 8000. Thermal stability analysis was conducted with an airflow rate of 20 mL min−1 and over a temperature range between 30 and 800 ℃, with an incremental heating rate of 25 ℃ min−1.
XRD analysis was used to identify crystallographic structures and phase purity of the synthesized control MNPs and BA-MNPs. XRD spectra of the respective samples are presented in Figure 2a. Inspection of Figure 2a reveals six significant diffraction peaks occurring at 2θ locations of 30.46, 35.78, 43.46, 53.88, 57.48, and 63.08°. These peaks were indexed and found to conform to the Miller indices of (220), (311), (400), (422), (511), and (440) crystal planes, respectively. These peaks are in good agreement with the standard card pattern (JCPDS No. 19-629) for pure Fe3O4 nanoparticles and correspond to a cubic structure with a space group of Fd-3m [52,53]. The diffraction patterns also confirm there were no other impurities present and indicate the capping agents were effective in preventing atmospheric oxidation.
Analysis of the XRD data revealed the BA-assisted synthesis process produced BA-MNPs with a smaller crystallite size. Table 1 presents the results of the crystallite size and crystallite size distribution study. Inspecting both Figure 2b and Table 1 reveals that both the size and size distribution of the BA-MNPs were smaller than the control MNPs produced without the presence of BA extract. Importantly, the crystallite size distribution of the BA-MNPs was very narrow when compared to the control MNPs. The narrow range between 16 and 20 nm of the BA-MNPs shows that the presence of BA extract in the synthesis process acts as a nanoparticle size regulator.
Sample | Scherrer (nm) | Rigaku SmartLab XRD | |
Size (nm) | Size distribution (nm) | ||
Control MNPs | 38 | 40 | 5–75 |
BA-MNPs | 16.03 | 18.2 | 16–20 |
The XRD data was first analyzed using the uniform deformation model (UDM) that was presented in Eq 9. Then, by plotting 4 sinθ along the x-axis and β(hkl) cosθ along the y-axis, a linear relationship was established, as seen in Figure 3a. From the graphical analysis, the micro-strain (slope) was found to be −2.6 × 10−3 and indicated the sample was under compression. The crystallite size determined from the y-intercept was found to be 9.11 nm, smaller than the Scherrer value of 38 nm. The second analysis method used the uniform stress deformation model (USDM) presented in Eq 11. The respective mechanical compliances for the BA-MNPs were determined and summarized in Table 2. The compliances were then used to determine the respective elastic modulus E(hkl) for each crystallographic plane, as seen in Table 3. Then, by plotting 4sinθ/E(hkl) along the x-axis and β(hkl) cos θ along the y-axis, a linear relationship was established, as seen in Figure 3b.
Material property | Value | |
Elastic compliances | S11 | 4.80 × 10−12 Pa−1 |
S12 | −1.37 × 10−12 Pa−1 | |
S44 | 10.13 × 10−12 Pa−1 | |
Stiffness parameters | C11 | 270 GPa |
C12 | 108 GPa | |
C44 | 98.7 GPa |
Crystal orientation | Modulus of elasticity, E (GPa) |
{220} | 221.06 |
{311} | 225.49 |
{400} | 208.33 |
{422} | 220.94 |
{511} | 220.44 |
From the graphical analysis, the stress was found to be −546.5 MPa, indicating the stress was compressive in nature. The micro-strain was estimated to be −2.47 × 10−3, and the crystallite size was found to be 9.17 nm. The third analysis method used the UDEDM presented in Eq 20. In this analysis, the UDEDM considers the influence of the strain energy density (u), which was calculated using Eq 19 and then substituted into Eq 20. Then, by plotting 4sinθ (2/E(hkl))1/2 along the x-axis and β(hkl)cosθ along the y-axis, a linear relationship was established, as seen in Figure 3c. From the graphical analysis, the energy density was found to be 710 KJ/m3, and the micro-strain was estimated to be −2.54 × 10−3. The analysis also found the crystallite size was 9.11 nm. The results of the three models are summarized in Table 4.
Sample | UDM | USDM | UDEDM | Average E (GPa) | |||||
BA-MNP | D(hkl) mm |
Strain ε (10−3) |
D(hkl) mm |
Stress σ (MPa) |
Strain ε (10−3) |
D(hkl) mm |
Energy density (KJ/m3) | Strain ε (10−3) | |
9.11 | −2.60 | 9.17 | −546.5 | −2.47 | 9.11 | 710 | −2.54 | 217.44 |
The D(hkl) value of the BA-MNPs estimated using the Debye-Scherrer equation was found to be 16.03 nm, slightly smaller than the 18.02 nm determined by the Rigaku SmartLab FP analysis. The D(hkl) values determined from UDM, UDSM, and UDEDM models are consistent with each other, as seen in Table 4. The mean D(hkl) value of 9.13 nm determined from these models is also consistent with the values obtained using the Debye-Scherrer method and Rigaku SmartLab FP analysis. Importantly, these sizes are comparable to those obtained by other researchers producing magnetite nanoparticles using similar chemical precipitation techniques, as seen in Table 5. In addition, the UDM, UDSM, and UDEDM models also give consistent negative lattice micro-strain values, thus indicating that the BA-MNPs are experiencing compressive stress and explaining the peak broadening seen in the XRD data [47]. Significantly, the presence of strain, whether compressive or tensile, often leads to peak broadening and even displacement of the 2θ peak position [59]. In addition, the synthesis technique itself contributes to the micro-strain experienced by the formed nanoparticles. For instance, during planetary ball-milling, nanoparticles experience high stresses, and the high residual stress generates large micro-strains [60]. Less stress-inducing chemical synthesis techniques like the co-precipitation method used in this study produce lower micro-strains in the generated MNPs [61,62].
Researcher | Average crystallite size D(hkl) (nm) | References |
Chaki, S.H., et al. (2015) | 6.58 | [55] |
Yusoff, A.H.M., et al. (2017) | 8–28 | [56] |
Kushwaha, P., et al. (2021) | 13–18 | [57] |
Ilyas, S., et al. (2019) | 22.5 | [58] |
Control sample | 5–75 | This study |
Williamson-Hall modeling | Mean 9.13 | This study |
Debye-Scherrer method | 16.03 | This study |
Rigaku SmartLab FP analysis | 18.20 | This study |
Data derived from the XRD diffraction pattern was used to determine the lattice parameter (a), as presented in Eq 4; the lattice volume (V) was calculated using Eq 5, as detailed in section 2. Then, the lattice parameter value was used to calculate the sample density ρ(XRD) using Eq 21. The calculated values for a, V, and ρ(XRD) are presented in Table 6.
Sample | Lattice constant a (Å) | Unit-cell volume V (Å3) | ρ(XRD) (gcm−3) |
BA-MNPs | 8.362 | 584.696 | 5.260 |
Once the ρ(XRD) was calculated, its value was substituted into Eqs 22–24 to calculate the longitudinal (VL), shear (VS), and mean (Vm) wave velocities, respectively. The calculated values for the respective wave velocities are presented in Table 7. Next, the wave velocity values of VL and VS were used to determine the longitudinal modulus (L) and the modulus of rigidity (G) using Eqs 25 and 26, respectively. Then, the longitudinal modulus and modulus of rigidity values were substituted into Eq 27 to calculate Poisson's ratio (σ). The calculated value of Poisson's ratio was found to be 0.2354, within the range between −0.9 and 0.5, which is consistent with the theory of isotropic elasticity. Finally, the bulk modulus of the sample was calculated using Eq 28. The values obtained in this study are similar to those derived by other researchers investigating the properties of magnetite nanoparticles [63,64]. The various calculated values, such as wave velocities, moduli, and Poisson's ratio, are summarized in Table 7.
Sample | ρ(XRD) (gcm−3) | VL (ms−1) | VS (ms−1) | Vm (ms−1) | L (GPa) | G (GPa) | σ | B (GPa) |
BA-MNPs | 5.260 | 7164.5 | 4136.4 | 4592.2 | 270.01 | 89.98 | 0.2354 | 159.87 |
The size, chemical composition, and morphology of the BA-MNPs produced in this study were characterized using a Zeiss Neon 40EsB FIBSEM and a Philips CM200 TEM. Inspection of the FIBSEM image presented in Figure 4a reveals that the synthesized BA-MNPs are highly agglomerated and need ultrasonic treatment to break up the large nanoparticle clusters [65]. Analysis of the FIBSEM images presented in Figure 4b also reveals the BA-MNPs have a spherical geometry with a mean diameter of 18 ± 5 nm. This size range is similar to the size estimates of 16.03 and 18.2 nm produced by the Scherrer method and the Rigaku SmartLab, respectively. In addition, the ±5 nm limits of the particle diameter also confirm the results of the Rigaku SmartLab analysis, which gave a size distribution range between 16 and 20 nm. Moreover, an EDS analysis was performed to confirm the formation and elemental composition of the BA-MNPs, as seen clearly in Figure 4c. Inspection of the EDS analysis reveals two strong Fe peaks located at 0.7 and 6.4 KeV, and a less pronounced Fe peak located at 7.1 KeV. Moreover, there is an intense peak located at 0.4 KeV, which represents the presence of oxygen. No other elements were detected in the EDS analysis. Thus, the ratio between Fe and O signals confirms the synthesized nanoparticles are indeed magnetite [66].
Figure 5 presents the results of the TEM investigation. Inspection of Figure 5a, b confirms the particle size and shape seen in the FIBSEM images presented in Figure 4. The images also confirm the tendency of the BA-MNPs to agglomerate. The agglomeration of the BA-MNPs can be attributed to the effects of van der Waals forces that facilitate nanoparticle attraction and clustering. Moreover, it is worth noting that the BA leaf extract contains the O–H functional group that has the potential to induce agglomeration [67]. In addition, SAED pattern analysis and high-resolution TEM image analysis was carried out to determine lattice plane spacing. Figure 5c shows the SAED pattern for the BA-MNPs sample. The diffraction rings were successfully indexed as (311), (220), (400), (422), (511), and (440). The indexing agreed with the XRD analysis, and similar results have been reported in the literature for magnetite nanoparticles [59,68]. The high-resolution TEM image shown in Figure 5d confirms the nanometer-scale polycrystalline structure of the BA-MNPs. The image analysis also found that the lattice spacing planar distance was 0.22 nm, confirming the results of the XRD data. Both the SAED pattern data and high-resolution image analysis confirm the well-defined crystalline structure of the BA-MNPs.
All the functional groups present in the BA extract and BA-MNPs were studied by FT-IR. Figure 6a shows a representative spectrum of each sample. The upper spectra in Figure 6a is the BA extract and shows bands located at 3357, 2918, 2849, 1731, 1604, 1516, 1161, and 1032 cm−1. The presence of these bands in the BA extract indicates the presence of flavonoids and phenolic functional groups in the BA extract, which have the potential to initially reduce metal ions to form metal nanoparticles and then stabilize the formed nanoparticles. A broad band was observed at 3357 cm−1 and was attributed to O–H stretching. Bands located at 2918 and 2849 cm−1 were assigned to C–H asymmetric and symmetric stretching vibrations of the methyl group. The band located at 1731 cm−1 was assigned to C = O stretching, and 1604 cm−1 was attributed to C–O vibrations. The band the located at 1516 cm−1 was assigned to Amide II linkage, and bands 1161 and 1032 cm−1 where attributed to coupled C–C and C–O vibrations of the phenolic group. The respective bands and assigned functional groups are summarized in Table 8.
Wavelength (cm−1) | Functional group | References |
3000–3600 | O–H functional group of phenol from BA-Raw | [69] |
2918, 2849 | C–H asymmetric and symmetric stretching vibrations of the methyl group | [70] |
1731 | C=O stretching due to xylan in hemicellulose | [71] |
1631 | C=O group of carboxylic acid | [72] |
1604 | C–O/aromatic C–C stretching |
[73] |
1516 | Amide II linkage | [74] |
1432 | Asymmetric and symmetric vibration of COO– | [74] |
1400–1466 | H–C–H bending vibrations | [75] |
551 | Fe–O bond vibration | [75] |
1000–1350 | Coupled C–C and C–O vibrations of the phenolic group | [76] |
The lower spectrum in Figure 6a is the BA-MNP sample and shows bands located at 3408, 1631, 1432, 1105, and 551 cm−1. The band located at 3408 cm−1 was attributed to O–H stretching and was shifted from 3357 cm−1 in the BA extract spectra. The shift suggests that the O–H functional group is acting as a reducing agent during the synthesis of the BA-MNPs [72,77,78]. The band located at 1631 cm−1 is believed to be the C = O group of carboxylic acid, which represents the carboxyl functional group (–COOH), with a slightly increased magnitude. The increase in magnitude of the band suggests the carboxyl functional group is acting as a capping agent during the formation of the BA-MNPs [78]. The band located at 1432 cm−1 was assigned to the asymmetric and symmetric vibration of COO–, while the band located at 1105 cm−1 was assigned to the C–O stretching vibration of phenolic groups. The final band in the spectra is located at 551 cm−1 and is assigned to Fe–O (stretching vibration) and confirms the formation of MNPs [72,79,80,81]. The respective bands and assigned functional groups are summarized in Table 8.
Figure 6b shows a representative Raman spectrum of a BA-MNP sample. The spectrum shows a strong peak located at 1283 cm−1, identified as the D-band of the BA-MNPs. The four peaks seen at 396, 471, 582, and 6555 cm−1 correspond to the vibration modes of the Fe–O bonds present in the BA-MNPs [82,83]. Also present are two strong peaks located at 220 and 286 cm−1, which were the result of an oxidation reaction that occurred during the Raman spectroscopy procedure [84]. Moreover, peaks located at 788 and 1040 cm−1 suggest organic compounds have induced surface modifications on the BA-MNPs. This result suggests the phytochemicals present in the BA extract can influence surface functionalization processes on the BA-MNPs [85,86]. Overall, the Raman spectrum is similar to other Raman studies reported in the literature and also confirms the formation of BA-MNPs [87].
A UV-visible spectrum of a representative BA-MNP sample is presented in Figure 7a. The spectra show a wide absorption range between 300 and 800 nm. However, there is a significant region of absorbance between 350 and 450 nm [88]. The absorption band in the UV range between 330 and 450 nm confirms the formation of BA-MNPs and is similar to reports discussed in the literature for plant-synthesized magnetite nanoparticles [89,90]. In addition, the UV-visible data was used to calculate the direct optical band-gap energy (Eg) for the BA-MNPs using Tauc's equation below:
αhv=K(hv−Eg)n | (30) |
In Eq 30, α = absorption coefficient, hʋ = incident photon energy, B is a material-dependent constant, Eg = optical band gap in electron-volts (eV), and n = an exponent that can take different values depending on the nature of the electronic transition. When the data from Eq 30 was plotted graphically with hʋ along the x-axis and αhʋ2 along the y-axis, the Tauc curve was produced, as seen in Figure 7b. Interception of the extrapolated linear part of the curve with the x-axis gave a band gap value of 2.63 eV. This band gap value is similar to those reported in the literature for magnetite nanoparticles synthesized by plants [88,89,90,91].
The results of TGA studies of the control MNPs and the BA-MNPs are presented in Figure 7c. Analysis of the control MNPs shows a steady decrease in sample weight of around 1.51% between 24 and 330 ℃. During the early part of this stage, water present in the capping vaporizes, leaving the salt-based capping still attached to the surface of the MNPs. However, at around 330 ℃, the salt-based coating has decomposed, leaving the bare MNPs. After 330 ℃, a small increase in weight occurs as the temperature and time increase. This small increase in weight was caused by the control MNPs undergoing surface oxidation [92]. TGA analysis of the BA-MNPs shows a two-step decomposition process over the temperature range, as seen in Figure 7c. During the first stage, between 24 and 120 ℃, there was a weight loss of 1.09% due to water evaporation and decomposition of some volatile phytochemicals [93,94]. During the second stage, between 150 and 400 ℃, there was a further weight loss of 2.99%. During this stage, salts and phytochemicals on the surface of the nanoparticles decomposed until only bare MNPs remained [95,96,97]. Overall, the thermal studies have shown the nanoparticle capping starts to decompose around 100 ℃; by 400 ℃, the capping agent was no longer present on the surface of the BA-MNPs.
LCA analysis was used to assess the environmental impact of the BA-MNPs produced in this study. The LCA analysis adhered to the procedures and guidelines as specified in ISO14040:2006 and ISO14044:2006 [98,99]. Table 9 presents a complete list of items used in preparing BA-MNPs.
Material/Process | Unit | Input |
Transportation | t-km | 0 |
Transportation of BA leaves | t-km | 0 |
Preparation of BA leaf extract | - | - |
BA leaves | g | 2 |
Milli-Q® water for cleaning BA leaves | mL | 100 |
Electrical energy for drying leaves | MJ | 0.04 |
Deionized water for supernatant | mL | 100 |
Electrical energy for heating | MJ | 2.29 |
BA-Mag NPs synthesis | - | - |
FeCl3 | g | 1.33 |
FeSO4·7H2O | g | 0.67 |
Electrical energy for heating | MJ | 0.72 |
NH4OH (28%) | g | 0.01 |
Ethanol for cleaning | mL | 50 |
Milli-Q® water for cleaning | mL | 200 |
Electrical energy for vacuum drying | MJ | 0.72 |
Total energy use | MJ | 13.99 |
Total water use | mL | 400 |
The life cycle inventory (LCI) for producing BA-MNPs from BA leaf extract was developed by deconstructing process inputs and emissions, as shown in Table 10. The procedure uses Eco-Invent datasets to model the environmental impact associated with the production and consumption of each input.
Category | Unit | Total | CO2 emissions |
Total water use | mL | 400 | N/A |
Total energy use | MJ | 3.77 | 0.0784 kg CO2 |
Total CO2 emissions | kg | - | 0.0784 kg CO2 |
From the LCI analysis, the overall CO2 emissions for the complete process were 0.0784 kg. The CO2 was generated from the energy needed for experimental processes like heating and vacuum drying and the production of precursor materials like FeCl3 and FeSO4·7H2O. In terms of CO2 emissions, the production of FeCl3, FeSO4·7H2O, ethanol, and NH4OH are minimal.
An assessment of the costs associated with new nanoparticle-based products is essential for determining their viability in transitioning from laboratory-based exploratory projects to pilot-based studies and then to large-scale industrial manufacturing for specific commercial applications [100]. The present cost analysis assesses the financial feasibility of the experimental procedure for producing BA-MNPs. Thus, a comprehensive analysis of the critical elements affecting the cost of production is crucial and needs to take in several factors, including: (1) availability of raw materials; (2) processing conditions and protocols; (3) processing equipment; (4) transportation; (5) labor; and (6) operational costs. In the context of the present exploratory study, the precursor BA leaves were readily available, other precursor material costs were negligible, and processing the BA-MNPs at the laboratory scale was straightforward and used standard in-house equipment. Therefore, the following analysis considers the driving expenses associated with synthesizing BA-MNPs. These expenses were determined through Eq 31.
BA−MNPsproductioncost=DC+HC+VDC | (31) |
In Eq 31, DC is the cost of drying samples after initial cleaning, HC is the heating cost during processing, and VDC is the vacuum drying cost after initial nanoparticle synthesis. Thus, the major expenses associated with BA-MNP production were drying, heating, and vacuum drying. The cost of each energy consumption activity was calculated using Eq 32.
Activitycost=Time(h)×EquipmentPower(kW)×electricitytariffinAustralia | (32) |
Other cost items are listed in Table 11. Inspection of Table 11 reveals the cost of producing BA-MNPs in this exploratory study was 1.26 USD/kg.
BA-Mag NPs synthesizing steps | Cost ($/kg) |
BA-MNP production cost (DC + HC + VDC) | 0.78 |
Cost of FeCl3 (IG)* | 0.14 |
Cost of FeSO4·7H2O (IG)* | 0.10 |
Cost of NH4OH (28%) | 0.01 |
Cost of EtOH | 0.23 |
Total cost | 1.26 |
**IG is the estimated cost of chemicals (www.chemsupply.com.au & www.sigmaaldrich.com). |
This exploratory study established that the Australian indigenous plant BA could be used in a co-precipitation method involving FeCl3 and FeSO4·7H2O to regulate the size and size distribution of the synthesized BA-MNPs. The mean particle size of the BA-MNPs was found to be 18.2 nm, and the narrow size distribution ranged from 16 to 20 nm. X-ray peak profile analysis using the UDM, USDM, and UDEDM models found the average crystallite size to be 9.13 nm. All the models showed that the crystal lattice of the BA-MNPs experienced a compressive stress of around 546 MPa. The models also found the BA-MNPs experienced an average micro-strain of 2.54 × 10−3. The study also used the XRD data to determine material properties, such as density (5.260 kg/m3), average Young's modulus of elasticity (217 GPa), and modulus of rigidity (90 GPa). Both FIBSEM and TEM microscopy studies confirmed the crystalline nature of the BA-MNPs and their size range. FT-IR and Raman spectroscopy verified the purity of the BA-MNPs and confirmed that the phytochemicals present in the BA extract acted as both reducing and capping agents. TGA analysis found the BA-MNPs were thermally stable over a temperature range of 24–800 ℃, with the capping decomposing at around 400 ℃. The laboratory cost for producing the BA-MNPs was found to be 1.26 USD/kg, and the overall emission of CO2 during the synthesis process was 0.0784 kg. Overall, the results show the co-precipitation method, using BA leaf extracts as a particle size regulator, can produce highly stable and pure BA-MNPs within a narrow size distribution. Future work will involve scaling up the current experimental procedure to produce commercial quantities of BA-MNPs. Future work will also involve refining the experimental procedure to tailor the properties of the BA-MNPs for specific applications, such as photocatalysts, bio-sensing and diagnosis carriers in medicine, and components in electronic and data storage devices.
The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.
Mr. A F M Fahad Halim would like to thank Murdoch University, Australia, for a scholarship to undertake his PhD studies. The authors would like to thank Mrs. Elaine Miller at the John de Laeter Center located at Curtin University for her assistance with FIBSEM imaging and the FEG-TEM facility located at Sophisticated Analytical Instrument Facility (SAIF), Indian Institute of Technology, Mumbai (Bombay), for their assistance with TEM microscopy.
Gerrad Eddy Jai Poinern conceived and planned the experiments, manuscript drafting, and proofreading. A F M Fahad Halim carried out the synthesis experiments, characterizations, data collection, literature search, manuscript drafting, and WH analysis. Derek Fawcett contributed to the interpretation of the results, manuscript drafting, literature search, and editing. Peter Chapman carried out the Raman spectroscopy. Rupam Sharma was involved in the TEM investigations. All authors provided critical feedback and helped shape the research, analysis, and manuscript final preparations.
The authors declare no conflict of interest.
[1] |
Chen Z, Wang S, Wang J, Xu K, Lei T, Zhang H, et al. (2021) Control strategy of stable walking for a hexapod wheel-legged robot. ISA transactions 108: 367‒380. https://doi.org/10.1016/j.isatra.2020.08.033. doi: 10.1016/j.isatra.2020.08.033
![]() |
[2] |
Liu J, Jayakumar P, Stein JL, Ersal T (2018) A nonlinear model predictive control formulation for obstacle avoidance in high-speed autonomous ground vehicles in unstructured environments. Vehicle system dynamics 56: 853‒882. https://doi.org/10.1080/00423114.2017.1399209. doi: 10.1080/00423114.2017.1399209
![]() |
[3] |
Lasi H, Fettke P, Kemper HG, Feld T, Hoffmann M (2014) Industry 4.0. Business & information systems engineering 6: 239‒242. https://doi.org/10.1007/s12599-014-0334-4. doi: 10.1007/s12599-014-0334-4
![]() |
[4] |
Zhong RY, Xu X, Klotz E, Newman ST (2017) Intelligent manufacturing in the context of industry 4.0: a review. Engineering 3: 616‒630, https://doi.org/10.1016/J.ENG.2017.05.015. doi: 10.1016/J.ENG.2017.05.015
![]() |
[5] |
Watkins CJCH, Dayan P (1992) Q-learning. Machine learning 8: 279‒292. https://doi.org/10.1007/BF00992698. doi: 10.1007/BF00992698
![]() |
[6] |
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521: 436–444, https://doi.org/10.1038/nature14539. doi: 10.1038/nature14539
![]() |
[7] | Mnih V (2013) Playing atari with deep reinforcement learning. arXiv preprint arXiv: 1312.5602. |
[8] | Wang Z, Schaul T, Hessel M, Hasselt H, Lanctot M, Freitas N (2016) Dueling network architectures for deep reinforcement learning. In International conference on machine learning, 1995‒2003. PMLR. |
[9] | Van Hasselt H, Guez A, Silver D (2016) Deep reinforcement learning with double q-learning. In Proceedings of the AAAI conference on artificial intelligence, 30. https://doi.org/10.1609/aaai.v30i1.10295. |
[10] | Hessel M, Modayil J, Van Hasselt H, Schaul T, Ostrovski G, Dabney W, et al. (2018) Rainbow: Combining improvements in deep reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence, 32. https://doi.org/10.1609/aaai.v32i1.11796. |
[11] | Schulman J, Levine S, Abbeel P, Jordan M, Moritz P (2015) Trust Region Policy Optimization. arXiv preprint arXiv: 1502.05477. |
[12] |
Xian B, Dawson DM, de Queiroz MS, Chen J (2004) A continuous asymptotic tracking control strategy for uncertain nonlinear systems. IEEE T Automat Contr 49: 1206‒1211. https://doi.org/10.1109/TAC.2004.831148. doi: 10.1109/TAC.2004.831148
![]() |
[13] | Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O (2017) Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06347. |
[14] |
Galicki M (2016) Finite-time trajectory tracking control in a task space of robotic manipulators. Automatica 67: 165‒170. https://doi.org/10.1016/j.automatica.2016.01.025. doi: 10.1016/j.automatica.2016.01.025
![]() |
[15] | Memarian F, Goo W, Lioutikov R, Niekum S, Topcu U (2021) Self-Supervised Online Reward Shaping in Sparse-Reward Environments. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2369‒2375. https://doi.org/10.1109/IROS51168.2021.9636020. |
[16] | Vecerik M, Hester T, Scholz J, Wang F, Pietquin O, Piot B, et al. (2017) Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards. arXiv preprint arXiv: 1707.08817. https://doi.org/10.48550/arXiv.1707.08817. |
[17] | Colas C, Sigaud O, Oudeyer PY (2018) Gep-pg: Decoupling exploration and exploitation in deep reinforcement learning algorithms. International conference on machine learning, 1039‒1048. PMLR. |
[18] | Conti E, Madhavan V, Petroski Such F, Lehman J, Stanley K, Clune J (2018) Improving exploration in evolution strategies for deep reinforcement learning via a population of novelty-seeking agents. Advances in neural information processing systems, 31. |
[19] | Lehman J, Stanley K O (2008) Exploiting open-endedness to solve problems through the search for novelty. In ALIFE, 329‒336. |
[20] |
Pugh JK, Soros LB, Stanley KO (2016) Quality diversity: A new frontier for evolutionary computation. Front Robot AI 3: 202845. https://doi.org/10.3389/frobt.2016.00040. doi: 10.3389/frobt.2016.00040
![]() |
[21] | Riedmiller M, Hafner R, Lampe T, Neunert M, Degrave J, Wiele T, et al. (2018) Learning by playing solving sparse reward tasks from scratch. In International conference on machine learning, 4344‒4353. |
[22] |
Wang C, Wang J, Wang J, Zhang X (2020) Deep-reinforcement-learning-based autonomous UAV navigation with sparse rewards. IEEE Internet of Things Journal 7: 6180‒6190. https://doi.org/10.1109/JIOT.2020.2973193. doi: 10.1109/JIOT.2020.2973193
![]() |
[23] |
Qi C, Zhu Y, Song C, Yan G, Xiao F, Zhang X, et al. (2022) Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle. Energy 238: 121703. https://doi.org/10.1016/j.energy.2021.121703. doi: 10.1016/j.energy.2021.121703
![]() |
[24] |
Li J, Shi X, Li J, Zhang X, Wang J (2020) Random curiosity-driven exploration in deep reinforcement learning. Neurocomputing 418: 139‒147. https://doi.org/10.1016/j.neucom.2020.08.024. doi: 10.1016/j.neucom.2020.08.024
![]() |
[25] | Liu IJ, Jain U, Yeh RA, Schwing A (2021) Cooperative exploration for multi-agent deep reinforcement learning. In International conference on machine learning, 6826‒6836. PMLR. |
[26] |
Xia Y, Lan M, Luo J, Chen X, Zhou G (2022) Iterative rule-guided reasoning over sparse knowledge graphs with deep reinforcement learning. Informa Process Manag 59: 103040. https://doi.org/10.1016/j.ipm.2022.103040. doi: 10.1016/j.ipm.2022.103040
![]() |
[27] |
Cai M, Hasanbeig M, Xiao S, Abate A, Kan Z (2021) Modular deep reinforcement learning for continuous motion planning with temporal logic. IEEE Robot Autom Lett 6: 7973‒7980. https://doi.org/10.1109/LRA.2021.3101544. doi: 10.1109/LRA.2021.3101544
![]() |
[28] | Roghair J, Niaraki A, Ko K, Jannesari A (2022) A vision based deep reinforcement learning algorithm for UAV obstacle avoidance. In Intelligent Systems and Applications: Proceedings of the 2021 Intelligent Systems Conference (IntelliSys) Volume 1, 115‒128. Springer International Publishing. https://doi.org/10.1007/978-3-030-82193-7_8. |
[29] |
Eoh G, Park TH (2021) Automatic curriculum design for object transportation based on deep reinforcement learning. IEEE Access 9: 137281‒137294. https://doi.org/10.1109/ACCESS.2021.3118109. doi: 10.1109/ACCESS.2021.3118109
![]() |
[30] | Christianos F, Schäfer L, Albrecht S (2020) Shared experience actor-critic for multi-agent reinforcement learning. Advances in neural information processing systems 33: 10707‒10717. |
[31] |
Hakak S, Alazab M, Khan S, Gadekallu TR, Maddikunta PKR, Khan WZ (2021) An ensemble machine learning approach through effective feature extraction to classify fake news. Future Generation Computer Systems 117: 47‒58. https://doi.org/10.1016/j.future.2020.11.022. doi: 10.1016/j.future.2020.11.022
![]() |
[32] | Hastings WK (1970) Monte Carlo sampling methods using Markov chains and their applications. |
[33] | Neal RM (2001) Annealed importance sampling. Statistics and Computing. |
[34] |
Silver D, Singh S, Precup D, Sutton RS (2021) Reward is enough. Artificial Intelligence 299: 103535. https://doi.org/10.1016/j.artint.2021.103535. doi: 10.1016/j.artint.2021.103535
![]() |
[35] |
Chen C, Hua Z, Zhang R, Liu G, Wen W (2020) Automated arrhythmia classification based on a combination network of CNN and LSTM. Biomed Signal Process Contr 57: 101819. https://doi.org/10.1016/j.bspc.2019.101819. doi: 10.1016/j.bspc.2019.101819
![]() |
[36] |
Liu X, Wen S, Hu Y, Han F, Zhang H, Karimi HR (2024). An active SLAM with multi-sensor fusion for snake robots based on deep reinforcement learning. Mechatronics 103: 103248. https://doi.org/10.1016/j.mechatronics.2024.103248. doi: 10.1016/j.mechatronics.2024.103248
![]() |
[37] |
Wen S, Shu Y, Rad A, Wen Z, Guo Z, Gong S (2025). A deep residual reinforcement learning algorithm based on Soft Actor-Critic for autonomous navigation. Expert Syst Appl 259: 125238. https://doi.org/10.1016/j.eswa.2024.125238. doi: 10.1016/j.eswa.2024.125238
![]() |
[38] | Azar AT, Sardar MZ, Ahmed S, Hassanien AE, Kamal NA (2023) Autonomous robot navigation and exploration using deep reinforcement learning with Gazebo and ROS. International Conference on Advanced Intelligent Systems and Informatics, 287‒299. Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-43247-7_26. |
[39] |
Haider Z, Sardar MZ, Azar AT, Ahmed S, Kamal NA (2024) Exploring reinforcement learning techniques in the realm of mobile robotics. Int J Autom Control 18: 655‒697. https://doi.org/10.1504/IJAAC.2024.142043. doi: 10.1504/IJAAC.2024.142043
![]() |
[40] |
Ahmed S, Azar AT, Tounsi M, Ibraheem IK (2023) Adaptive control design for Euler–Lagrange systems using fixed-time fractional integral sliding mode scheme. Fractal and Fractional 7: 712. https://doi.org/10.3390/fractalfract7100712. doi: 10.3390/fractalfract7100712
![]() |
[41] |
Jahanshahi H, Zhu ZH (2024) Review of Machine Learning in Robotic Grasping Control in Space Application. Acta Astronaut 15. https://doi.org/10.1016/j.actaastro.2024.04.012. doi: 10.1016/j.actaastro.2024.04.012
![]() |
1. | Adina-Elena Segneanu, Gabriela Vlase, Catalin Nicolae Marin, Titus Vlase, Crina Sicoe, Daniel Dumitru Herea, Maria Viorica Ciocîlteu, Ludovic-Everard Bejenaru, Anca Emanuela Minuti, Camelia-Mihaela Zară, Vlad Socoliuc, Cristina Stavila, Cornelia Bejenaru, Wild grown Portulaca oleracea as a novel magnetite based carrier with in vitro antioxidant and cytotoxicity potential, 2025, 15, 2045-2322, 10.1038/s41598-025-92495-7 |
Sample | Scherrer (nm) | Rigaku SmartLab XRD | |
Size (nm) | Size distribution (nm) | ||
Control MNPs | 38 | 40 | 5–75 |
BA-MNPs | 16.03 | 18.2 | 16–20 |
Material property | Value | |
Elastic compliances | S11 | 4.80 × 10−12 Pa−1 |
S12 | −1.37 × 10−12 Pa−1 | |
S44 | 10.13 × 10−12 Pa−1 | |
Stiffness parameters | C11 | 270 GPa |
C12 | 108 GPa | |
C44 | 98.7 GPa |
Crystal orientation | Modulus of elasticity, E (GPa) |
{220} | 221.06 |
{311} | 225.49 |
{400} | 208.33 |
{422} | 220.94 |
{511} | 220.44 |
Sample | UDM | USDM | UDEDM | Average E (GPa) | |||||
BA-MNP | D(hkl) mm |
Strain ε (10−3) |
D(hkl) mm |
Stress σ (MPa) |
Strain ε (10−3) |
D(hkl) mm |
Energy density (KJ/m3) | Strain ε (10−3) | |
9.11 | −2.60 | 9.17 | −546.5 | −2.47 | 9.11 | 710 | −2.54 | 217.44 |
Researcher | Average crystallite size D(hkl) (nm) | References |
Chaki, S.H., et al. (2015) | 6.58 | [55] |
Yusoff, A.H.M., et al. (2017) | 8–28 | [56] |
Kushwaha, P., et al. (2021) | 13–18 | [57] |
Ilyas, S., et al. (2019) | 22.5 | [58] |
Control sample | 5–75 | This study |
Williamson-Hall modeling | Mean 9.13 | This study |
Debye-Scherrer method | 16.03 | This study |
Rigaku SmartLab FP analysis | 18.20 | This study |
Sample | Lattice constant a (Å) | Unit-cell volume V (Å3) | ρ(XRD) (gcm−3) |
BA-MNPs | 8.362 | 584.696 | 5.260 |
Sample | ρ(XRD) (gcm−3) | VL (ms−1) | VS (ms−1) | Vm (ms−1) | L (GPa) | G (GPa) | σ | B (GPa) |
BA-MNPs | 5.260 | 7164.5 | 4136.4 | 4592.2 | 270.01 | 89.98 | 0.2354 | 159.87 |
Wavelength (cm−1) | Functional group | References |
3000–3600 | O–H functional group of phenol from BA-Raw | [69] |
2918, 2849 | C–H asymmetric and symmetric stretching vibrations of the methyl group | [70] |
1731 | C=O stretching due to xylan in hemicellulose | [71] |
1631 | C=O group of carboxylic acid | [72] |
1604 | C–O/aromatic C–C stretching |
[73] |
1516 | Amide II linkage | [74] |
1432 | Asymmetric and symmetric vibration of COO– | [74] |
1400–1466 | H–C–H bending vibrations | [75] |
551 | Fe–O bond vibration | [75] |
1000–1350 | Coupled C–C and C–O vibrations of the phenolic group | [76] |
Material/Process | Unit | Input |
Transportation | t-km | 0 |
Transportation of BA leaves | t-km | 0 |
Preparation of BA leaf extract | - | - |
BA leaves | g | 2 |
Milli-Q® water for cleaning BA leaves | mL | 100 |
Electrical energy for drying leaves | MJ | 0.04 |
Deionized water for supernatant | mL | 100 |
Electrical energy for heating | MJ | 2.29 |
BA-Mag NPs synthesis | - | - |
FeCl3 | g | 1.33 |
FeSO4·7H2O | g | 0.67 |
Electrical energy for heating | MJ | 0.72 |
NH4OH (28%) | g | 0.01 |
Ethanol for cleaning | mL | 50 |
Milli-Q® water for cleaning | mL | 200 |
Electrical energy for vacuum drying | MJ | 0.72 |
Total energy use | MJ | 13.99 |
Total water use | mL | 400 |
Category | Unit | Total | CO2 emissions |
Total water use | mL | 400 | N/A |
Total energy use | MJ | 3.77 | 0.0784 kg CO2 |
Total CO2 emissions | kg | - | 0.0784 kg CO2 |
BA-Mag NPs synthesizing steps | Cost ($/kg) |
BA-MNP production cost (DC + HC + VDC) | 0.78 |
Cost of FeCl3 (IG)* | 0.14 |
Cost of FeSO4·7H2O (IG)* | 0.10 |
Cost of NH4OH (28%) | 0.01 |
Cost of EtOH | 0.23 |
Total cost | 1.26 |
**IG is the estimated cost of chemicals (www.chemsupply.com.au & www.sigmaaldrich.com). |
Sample | Scherrer (nm) | Rigaku SmartLab XRD | |
Size (nm) | Size distribution (nm) | ||
Control MNPs | 38 | 40 | 5–75 |
BA-MNPs | 16.03 | 18.2 | 16–20 |
Material property | Value | |
Elastic compliances | S11 | 4.80 × 10−12 Pa−1 |
S12 | −1.37 × 10−12 Pa−1 | |
S44 | 10.13 × 10−12 Pa−1 | |
Stiffness parameters | C11 | 270 GPa |
C12 | 108 GPa | |
C44 | 98.7 GPa |
Crystal orientation | Modulus of elasticity, E (GPa) |
{220} | 221.06 |
{311} | 225.49 |
{400} | 208.33 |
{422} | 220.94 |
{511} | 220.44 |
Sample | UDM | USDM | UDEDM | Average E (GPa) | |||||
BA-MNP | D(hkl) mm |
Strain ε (10−3) |
D(hkl) mm |
Stress σ (MPa) |
Strain ε (10−3) |
D(hkl) mm |
Energy density (KJ/m3) | Strain ε (10−3) | |
9.11 | −2.60 | 9.17 | −546.5 | −2.47 | 9.11 | 710 | −2.54 | 217.44 |
Researcher | Average crystallite size D(hkl) (nm) | References |
Chaki, S.H., et al. (2015) | 6.58 | [55] |
Yusoff, A.H.M., et al. (2017) | 8–28 | [56] |
Kushwaha, P., et al. (2021) | 13–18 | [57] |
Ilyas, S., et al. (2019) | 22.5 | [58] |
Control sample | 5–75 | This study |
Williamson-Hall modeling | Mean 9.13 | This study |
Debye-Scherrer method | 16.03 | This study |
Rigaku SmartLab FP analysis | 18.20 | This study |
Sample | Lattice constant a (Å) | Unit-cell volume V (Å3) | ρ(XRD) (gcm−3) |
BA-MNPs | 8.362 | 584.696 | 5.260 |
Sample | ρ(XRD) (gcm−3) | VL (ms−1) | VS (ms−1) | Vm (ms−1) | L (GPa) | G (GPa) | σ | B (GPa) |
BA-MNPs | 5.260 | 7164.5 | 4136.4 | 4592.2 | 270.01 | 89.98 | 0.2354 | 159.87 |
Wavelength (cm−1) | Functional group | References |
3000–3600 | O–H functional group of phenol from BA-Raw | [69] |
2918, 2849 | C–H asymmetric and symmetric stretching vibrations of the methyl group | [70] |
1731 | C=O stretching due to xylan in hemicellulose | [71] |
1631 | C=O group of carboxylic acid | [72] |
1604 | C–O/aromatic C–C stretching |
[73] |
1516 | Amide II linkage | [74] |
1432 | Asymmetric and symmetric vibration of COO– | [74] |
1400–1466 | H–C–H bending vibrations | [75] |
551 | Fe–O bond vibration | [75] |
1000–1350 | Coupled C–C and C–O vibrations of the phenolic group | [76] |
Material/Process | Unit | Input |
Transportation | t-km | 0 |
Transportation of BA leaves | t-km | 0 |
Preparation of BA leaf extract | - | - |
BA leaves | g | 2 |
Milli-Q® water for cleaning BA leaves | mL | 100 |
Electrical energy for drying leaves | MJ | 0.04 |
Deionized water for supernatant | mL | 100 |
Electrical energy for heating | MJ | 2.29 |
BA-Mag NPs synthesis | - | - |
FeCl3 | g | 1.33 |
FeSO4·7H2O | g | 0.67 |
Electrical energy for heating | MJ | 0.72 |
NH4OH (28%) | g | 0.01 |
Ethanol for cleaning | mL | 50 |
Milli-Q® water for cleaning | mL | 200 |
Electrical energy for vacuum drying | MJ | 0.72 |
Total energy use | MJ | 13.99 |
Total water use | mL | 400 |
Category | Unit | Total | CO2 emissions |
Total water use | mL | 400 | N/A |
Total energy use | MJ | 3.77 | 0.0784 kg CO2 |
Total CO2 emissions | kg | - | 0.0784 kg CO2 |
BA-Mag NPs synthesizing steps | Cost ($/kg) |
BA-MNP production cost (DC + HC + VDC) | 0.78 |
Cost of FeCl3 (IG)* | 0.14 |
Cost of FeSO4·7H2O (IG)* | 0.10 |
Cost of NH4OH (28%) | 0.01 |
Cost of EtOH | 0.23 |
Total cost | 1.26 |
**IG is the estimated cost of chemicals (www.chemsupply.com.au & www.sigmaaldrich.com). |