Convolutional neural network with group theory and random selection particle swarm optimizer for enhancing cancer image classification

1.   Introduction

Rice (Oriza sativa L.) is the main food staple for more than half of the world's population, particularly those living in some of the most populous countries, such as China, India and Japan ^[1,2,3]. Humans have a long history of cultivating rice crops; Rice was first domesticated approximately 9000 years ago by people living in the region of the Yangtze River valley in China ^[4]. For the Food and Agriculture Organization (FAO) of the United Nations (UN), world rice production forecasts an overall increase of 10.3 million tons per year to a new maximum of 769.9 million tons in 2018 ^[5], at normal growing conditions. This growth in production output driven by human consumption, which would decrease other uses, such as industrial and feed uses. Due population growth, global per capita food consumption showed an increase (0.4%) from 53.7 kg in 2017, to 53.9 kg in 2018. In terms of global rice consumption, by 2020 this amounted to a total of 379.035 million tons for the major consuming countries (China, India, Indonesia, Bangladesh, Vietnam, Philippines, Myanmar, Thailand, Japan and Brazil) ^[6]. Rice cultivation activity currently faces a series of challenges in order to meet current and projected future global consumption rates. Of most importance is the limited opportunity for expansion due to outright decline of available arable land ^[7,8], the increasingly limited availability of fresh water for irrigation due to climate change, increased industrial activity and urban freshwater usage ^[7,9,10,11]. Moreover, several additional factors have been identified as key contributors to declining rice crop yields such as the narrow genetic background of rice plants, inadequate soil drainage, which contributes to increased salinity and alkalinity over time, soil nutrient and organic matter deficiencies, and further soil quality decline from fertilizer and pesticide overuse ^[7,12]. Finally, the prevalent lack of strong research extension-farmer relationships, farmer organizations and public-private partnerships limits the capacity of producers to overcome these challenges ^[7].

Due to the urgent imminent requirement of supplying the growing world population with a sufficient volume of rice crop, it is important to know the development cycles of this plant, as well as the factors, both climatic and environmental, that limit its development and yield. In addition, it is important to study the influence of environmental factors, such as the presence of heavy metals or other contaminants, in the nutritional profile and potential toxicity of cultivated rice. It is also necessary to consider all the methodologies and technologies available for crop analysis such as the mechanisms of pollutants uptake and translocation in rice, phenology, canopy reflectance spectra, remote sensing, near-surface imaging spectroscopy and, of particular interest to this work, hyperspectral imaging ^[13].

The tangible and potential improvements introduced by Information and Communications Technologies (ICTs), particularly from the disruptive fields of machine learning and digital signal processing and including widely-deployed acquisition technologies such as the Internet of Things (IoT) ^[14,15], biosensors ^[16] and wireless sensor networks ^[17], have impacted such a wide variety of fields that the UN has stated their confidence that these technologies have changed the modern world, and possess the potential to continue to do so in a manner that makes them key to fulfilling their Sustainable Development Goals ^[18]. These technologies have introduced tangible benefits in various precision agriculture applications for rice cultivation applications ^{[19,20,21,22]}. Additionally, these technologies have enabled improvements in automated crop monitoring ^[23], efficient production input distribution ^[24], and crop quality evaluation tasks ^[25]. Several of these applications have been academically proposed but have faced difficulties in reaching sufficiently large-scale deployments as to shift the established agricultural production paradigms. Precision agriculture implied the use of sensors and advanced imaging technique to obtain data to monitoring the health status of the crops ^[26]. Hyperspectral imaging systems have gained notable relevance due to the advanced development of remote sensing systems with higher spectral and spatial resolution and have been used as a tool in precision agriculture in studies of morphological and physical stages of different crops however does not have been fully exploited. This spectral sensing technique allows the study of crop biophysical and biochemical properties, soil characteristics and crops classification as well to monitoring large fields of cultivation. Due to the advances in optoelectronics and data acquisitions systems HSI is now more accessible and allows the access to low cost and small size hyperspectral sensors. ^[27]. Additionally, a plural development of algorithms to work with high amount of data to extract information from the spectral signature of the materials with advanced techniques such as linear regression, nonlinear regression, and combined methods that incorporate machine learning techniques such as Support Vector Machines, neural networks and others ^{[26,28,29,30]}. Recent studies use the spectral measurements of various crops to infer important magnitudes such as nitrogen (N) content ^[31], landcover classification of cultivated areas ^[32] and individual plants ^[33], detection of plant diseases ^[34], detection of pest presence ^[35] and damage to plants ^[36] and various others, which creates opportunities for improved yield and process efficiency in rice cultivation through their use in smart remote sensing applications.

The main contribution of this work is to provide the community with an up-to-date survey of the most relevant works regarding the use of hyperspectral imaging sensing technology and hyperspectral image processing algorithms for the estimation and monitoring of various aspects of the rice cultivation process without the requirement of a field-wide deployment of individual sensors. Additionally, we provide a comparison of the signal processing methodologies used in the most relevant studies and present the main challenges for application deployments in the field. The methodologies proposed in these studies can potentially contribute to the cost reduction of measurement systems in precision agriculture operations, and alleviate the human labor requirements for the processes of assessing the health and expected productivity of rice plants in all its stages, from seeding to harvest on the field. Advances made towards alleviating these challenges are of special interest to all involved in the development and deployment of innovative precision agriculture applications. Particularly, funding agencies for rice producers such as banks, insurance companies, private investors, and other entities who would benefit from more accurate return of investment (ROI) estimates. With more informed yield estimates, more accurate risk assessment studies can be conducted by these agencies and more producers can potentially benefit from increased availability of funding sources for their activities. In this manner, it is our hope that this work serves as a starting point for future research efforts.

2.   Constraining factors in rice cultivation

Rice production, like other types of crops, have been adapting farming methods to deal with problems connected to climate change, the growing demand for food, and the limitation of the amount of land for cultivation. The key challenges for rice cultivation are: sea water intrusion which is a problem of salinity in landscape areas (particularly sea level rise zones in Asia), deforestation, variability in precipitations patterns and water scarcity that requires a dynamic between availability and management of this resource ^[37]. Those problems require the use of new rice species that are resistant as well as new technologies to support monitoring, predictive analysis and water saving systems which are commonly linked to precision agriculture. Precision agriculture is a set of signal processing techniques and specialized machinery, sensors, actuators and various information systems in order to optimize crop production in agricultural systems ^[38]. Thus, the main object of study of this field is to further increase the effectiveness of agricultural activity by incorporating additional variables of interest into the decision-making systems from sensor technology developments, improving decision making processes through various algorithms, and new innovations in actuator technology. In this manner, the potential yields of the agricultural process are maximized by continuously ensuring the optimal conditions for adequate plant growth and development. In most ecosystems around the world, Photosynthesis is one of the most important processes for capturing energy from sunlight. The balance of the photosynthetic and respiration processes in plants is crucial for understanding their growth and productivity expectations, as well as the overall carbon cycle of ecosystems ^[39,40].

2.1.   Chlorophyll and Nitrogen

Chlorophyll is an important indicator of the photosynthetic capacity and vegetation stress in plants, changing its concentration in response to environmental conditions and solar radiation ^[41,42]. In previous studies, plant chlorophyll content has been identified to describe a close relationship to the gross food and biomass production capabilities of plants ^[43,44]. Further, there exists an established correlation between the measured N levels present in plant leaves and its expected productivity levels ^[45]. Thus, N is considered as a limiting element for plant growth since is the key nutrient parameter determining the photosynthetic functioning and productivity in rice crops ^[46]. N deficiency in plants can lead to lower chlorophyll content, lower photosynthetic assimilation, less biomass production, and reduced grain yield. On the other hand, a high content of N can improve chlorophyll density, which results in higher levels of photosynthetic activity and, by extension, productivity levels ^[47]. However, increasing N levels through external production inputs, such as fertilizers, can cause a series of environmental pollution issues and even diminished yields when fertilization become excessive ^[48]. The excess reactive N, derived from over and/or improper application of N fertilizers, may produce detrimental effects on public health and ecosystems. Excess N deposition in atmosphere considered recognition as a potent contributor to global warming and stratospheric ozone depletion ^[49].

2.2.   Carbon cycle

In addition, the study of the vegetation ecosystem carbon cycle attracts considerable interest to assess carbon sequestration capability ^[23,50]. In the face of rising global atmospheric CO₂ levels that contribute to climate change, elevated carbon assimilation in crop plants has promising effects on both crop yields and sequestration capabilities to aid in lowering the overall carbon levels in the atmosphere ^[51].

2.3.   Soil nutrients

A key factor to consider in precision agriculture is the amount of nutrients present in the soil. Nutrient availability in soils play an important role in agricultural productivity, food security and agroecological sustainable development ^[52,53]. Parameters such as soil moisture, organic carbon, organic matter, heavy metals and soil nutrient contents are common soil properties and should be consider in prediction models for reducing soil nutrient loss and improving soil fertilization management practices, based on more robust and more sensitive analytical techniques, to complement plant information. To this end, specific wavelength ranges in aerial hyperspectral images have been used to characterize the nutrient content and distribution over regional scales ^[53].

2.4.   Heavy metal pollution

Due to the rapid development of the industry economy in adjacency to agricultural land, rice paddies have been subjected to heavy metal pollution in some areas ^[27,54,55]. Some metals as cadmium (Cd), arsenic (As), and lead (Pb) are of great concern. It is known, that in China, for example, nearly 20 million hectares of cultivated land is contaminated with metals, resulting in about 12 million tons of contaminated grains which translate to about 2.4 billion US dollars of economic loss per year. Heavy metal pollution is becoming a worldwide concern to both agriculture and human health. One of the most contaminant elements is Cd, characterized as a toxic heavy metal, because either can contaminate the soil with a high mobility in living organisms, or have toxic effects on rice expose to this metal. As of constants applications of N fertilizers in agriculture, many soils become more acidic allowing Cd being easily absorbed by plants ^[13]. Cd is transferred from soil to rice and accumulates in rice plants and grains and then is enriched in the human body thought the food chain, thereby threatening human health. The high mobility of Cd causes effect on rice plants and crops exposed to this metal by interfering with the physiological activities of plants, such as photosynthesis, gaseous exchange, and nutrient absorption, to cause reduction in plant growth and dry matter accumulation ^[56]. In addition to crop health, heavy metal pollution has been demonstrated to negatively impact human health, and to cause effects such as anemia, cancer, heart failure, hypertension, cerebral infarction, proteinuria, severe lung damage, eye cataract formation, osteoporosis, emphysema, and renal insufficiency ^[13,57,58].

2.5.   Water management

Shifting weather patterns as a result of imminent climate change is a growing concern for government agencies worldwide due to the potential shifting in water availability around the world ^[59,60]. Failure to adapt proper water management practices can have significant adverse effects on the food security of a given region. For the purposes of properly assessing and managing both the quantity and quality of available water resources, hyperspectral imaging applications have been devised in various regions ^{[61,62,63,64]}. On the subject of hyperspectral imaging applications for water management precision agriculture, there is a growing interest in monitoring and managing drought stress to maintain high production levels in intensive crop farming applications ^{[65,66,67,68,69]}. Through a combination of drought-tolerant cultivars and precise water resource availability monitoring, irrigation practices can be optimized in order to ensure the continued increase in yields and food security for communities in drought-prone areas.

3.   Hyperspectral imaging

Hyperspectral images provide valuable, spatially-resolved information for remote sensing applications in hundreds of individual wavelengths beyond the visible spectrum. With the increase of available spectral information about their targets, it is possible to perform accurate classification of objects and/or materials within a spatial scene of interest. This ability has demonstrated to be of use in fields such as studies of the atmosphere, vegetation, agriculture and coastal environments. Hyperspectral imaging has important applications in precision agriculture, where the health status of crops in different growth stages can be assessed from their spectral signatures.

Hyperspectral imaging sensors are spatial measurement devices that capture the spectral behavior of a scene of interest in the form of many simultaneous digital images, each representing a narrow spectral region across a continuous or discontinuous spectrum ^[70]. When a given material is exposed to a light source of a known spectral bandwidth, it emits, absorbs, and/or reflects specific portions of the electromagnetic spectrum according to its composition. This material-specific response is known as a material's spectral signature ^[70]. As illustrated in Figure 1, this information is stored in a cubic data structure, where individual spectral bands are "stacked" on top of each other, ordered by their individual wavelengths.

With this information, an observation of the measured spectral responses allows for the classification of different materials, or the observation of specific compositional qualities of interest in biological subjects. This technology has been applied to water resources and flood management, food quality assessment, medical diagnosis, military applications and various others ^{[29,71,72,73]}. Of particular importance to the current work, the advantages introduced by hyperspectral image analysis in precision agriculture are crucial for monitoring the effects of various factors on the spectral response of plant tissues such as fertilization ^[74], micronutrient content ^[75], pest damages ^[36], pollutant uptakes ^[76] or extreme conditions, such as droughts and flooding in localized regions ^[77,78]. From the measured variations in spectral responses, further processing of spectral data can determine the effects of these factors on the safety and productivity of crops. In order to provide images with a high spectral resolution, hyperspectral imaging devices' construction requires complex optomechanical components, which limits their deployability due to increased weight, volume and power requirements. As illustrated in Figure 2, these devices can be mounted on various platforms such as satellites ^[79], aircraft ^[80], Unmanned Aerial Vehicles (UAV) ^[81], or handheld devices which can be used in the field or in laboratory settings. Due to the fact that each platform provides different tradeoffs regarding spatial resolution, spectral resolution, measurement noise, coverage and/or deployment costs, the selection of the optimal imaging platform is application specific. Hyperspectral imaging enables researchers and plant breeders to analyze various traits of interest like nutritional values in high throughput ^[82]. For these authors, it is necessary to obtain an optimal design of a reliable calibration model, linking the measured spectra with the investigated traits. It is important to consider, the inclusion of samples from several time and locations to improve the predictions of the investigated nutrient trait at small calibration set sizes. However, some models obtained have some limitations because they work only in certain environments and are not transferable to others, for this reason they should be upgraded with new calibration data to enable a reliable response of desired traits.

Another important aspect to consider in agriculture in general, and specially in rice cultivation, for setting calibration of remote sensing hyperspectral imaging, is the spatial distribution of soil nutrients which depends mostly on field sampling and laboratory analysis, but in some cases, result inefficient and time consuming ^[53]. Nevertheless, applying variables to predict soil nutrients is a key means of clarifying their spatial variations; is in our opinion, to select carefully representative sampling field sites followed by the proper laboratory analysis of selected parameters e.g. N, phosphorus, and soil potassium contents of soil due to they are closely related to nutrient cycling for growth and fertilizer application in human activities, to support all the calibration models to get more accurate hyperspectral imaging data.

In the context of precision agriculture, the introduction of hyperspectral imaging presents some important advantages as a noninvasive observation and/or monitoring methodology that does not interfere with crops' growth process. In addition, a single aerial hyperspectral image provides a higher volume of detailed information about the developmental progress and chemical composition of both the plants and soil in a reduced labor-intensive manner compared to laboratory analysis of field samples. With the objective of estimating productivity levels, previous work on the spectral analysis of agriculture hyperspectral images has determined mathematical models that describe the relationships between the spectral behavior of rice, considering both individual plants and canopies and their chlorophyll, N and total carbon content ^[23,49,83]. Similar studies have been conducted for additional food crops such as wheat ^[84], corn ^[85,86], barley ^[82] and others ^{[87,88,89,90,91]}.

In addition, hyperspectral images have been used to estimate various agriculturally relevant soil properties such as carbon content ^[92,93,94], heavy metal content ^[95,96], salinity ^[97], pH ^[98] and overall fertility ^[99]. Moreover, previous studies have demonstrated the possibility of hyperspectral image-based methodologies for the detection of fungal organisms and toxins in both crop plants and food products, which hinder the productivity of plants ^{[100,101,102,103]}, contribute to additional levels of food waste ^[104,105], and have adverse effects on human health when consumed ^[106,107]. Finally, hyperspectral imaging has been employed for detecting the presence of heavy metal contaminants in the soil ^[106] and crop plants themselves ^[108].

4.   Hyperspectral imaging for rice cultivation applications

In this section, we highlight and summarize the most relevant contributions of hyperspectral imaging rice cultivation applications in different stages of plant growth, for various quantities of interest. After reviewing 168 articles of the most relevant literature, the most significant studies in hyperspectral rice studies includes a total of 26 (49%) studies aimed at Chlorophyll and/or N estimation, 11 (19%) for various seed studies, 11 (19%) for biomass and vegetation quantity estimation and 10 (17%) for detection of pests, diseases and various pollutants. The distribution of studies is illustrated in Figure 4b.

Furthermore, the significant contributions in literature are organized, according to their objectives, in Tables 2 to 5. Table 2 lists the surveyed studies relative to biomass and vegetation estimation in rice crops or landcover classification of rice growing sites; Table 3 summarizes significant work concerning chlorophyll and N content estimation; studies of diseases, pests, heavy metals and other pollutants' effects on rice crops are summarized in Table 4; studies focused in various seed properties, lodging and others variables are presented in Table 5.

For rice cultivation, the studies in Tables 2-5 contemplate different rice species, inferred magnitudes such as total N content, chlorophyll, seed type, viability and/or origin, disease and plague presence, herbicide toxicity, carbon content, and the spectral region in which these properties manifest as a measurable spectral response in plants or cultivated rice. Nevertheless, only 20% of the reviewed literature examine soil variables and their direct relation with rice plant growth and health ^{[31,32,33,49,76,109,110,111,112,113,114,115]}, especially to set up a single data base which appraise both to establish accurate calibration model for remote sensing imaging predictions. As observed in Tables 2 to 5, the majority of rice studies are executed in Asian locations, particularly in mainland China, as is illustrated in Figure 3. Further, the predominant acquisition methodology in the surveyed literature is manual acquisition, where spectral measurements are captured in the field by handheld spectral measurement devices. It is important to know that, while most handheld devices measure a single spatially averaged spectrum (depending on the instrument's Field of View), the insights obtained from the studies on collected spectra can be applied to spatial HSIs if/when available. The justification for the use of handheld spectrometers is twofold: first, there is a limited number of satellite missions with onboard measurement devices that are relevant to agricultural applications in adequate spatial resolutions with public data accessibility. Second, for the purposes of a single or small amount of studies, there is a significant decrease in cost and planning time requirements when using handheld devices in comparison to mobile acquisition platforms, such as UAVs. Figure 1a illustrates the distribution of employed sensor platforms for hyperspectral imaging of rice, showing a clear majority of handheld measurement devices, followed by UAV-mounted sensors. Additionally, the majority of the studies are performed on a nonstandardized plurality of individual rice cultivars, which have not been spectrally catalogued according to their visible or NIR spectral responses for future reference.

This problem is further compounded by the continuous selective breeding of improved rice cultivars with attractive features for individual regions such as resistance to heat/cold, pests, presence of pollutants or increased productivity for a given amount of fertilizer input. For this reason, signal processing methodologies for precision agriculture of rice crops must be robust enough to accommodate for slight variations in spectral responses from newly introduced rice cultivars, in addition to various factors that are vulnerable to imminent shifting climate conditions, such as drought/flood responses and soil quality ^[55].

A large part of recent studies is focused on the use of hyperspectral reflectance data. That is, the evaluation of the spectral behavior of an incident light's fraction of reflected photons off an object of study. However, a subset of studies includes the fluorescent spectrum, which are obtained by capturing the spectrum of the light emitted by the target of interest when exposed to an ultraviolet (UV) light source. Due to the low intensity of the emitted light, fluorescence studies tend to be limited to laboratory environments, where the reflectance component of other light sources can introduce additive noise to the fluorescence spectral measurements ^{[116,117,118,119,120]}.

While there exist various hardware configurations for hyperspectral image acquisition such as Whisk Broom ^[121], Push Broom ^[122] and Computer Tomography Imaging Spectrometers ^[123], the current study is focused on the study of the measured spectral response data. Table 1 summarizes the most commonly used sensors in studies utilizing multispectral and/or hyperspectral imaging devices for rice cultivation applications, according to their operating principles.

For applications where hyperspectral reflectance is measured in the field, the incident light from the sun is considered. This presents an advantage in crop field applications, where the use of the sun as a light source reduces the complexity of the acquisition platform. Conversely, in laboratory measurements, a broadband illumination source is employed ^{[103,124,125,126]}. Across the reviewed literature, we found that a total of 37 studies (63%) performed measurements in the field, while 15 (25%) performed lab measurements with an artificial light source. The proportion of these studies, is illustrated in Figure 5, including the 7 studies that make use of publicly available spectral measurements.

Tables 2 to 5 summarize the most relevant work performed on the application of hyperspectral imaging for rice cultivation, highlighting the main objects of study and the extent of the spectral ranges evaluated in each work. These Tables present the acquisition method, spectral region, rice species use in each work and the region of the study, which can be used as a reference of local climate conditions. It is important to highlight that the local soil conditions were only taken into consideration for the proposed methodologies in two individual studies ^[33,49]. With the exceptions of ^[127] (which employed a publicly available dataset from the NEON 2013 Imaging Campaign in California ^[128]), and ^{[32,109,112,113,114,129]} (which employed publicly available Landsat and/or MODIS measurements), all the surveyed literature included their individual data collection campaigns as part of their contributions.

4.1.   Biomass and vegetation estimation

Table 2 summarizes the most relevant works in hyperspectral imaging applications for biomass and vegetation estimation. These studies allow for remote detection of individual rice plants ^[33], common weeds ^[149] and large-scale landcover identification of the presence of rice paddies ^{[112,114,129]}. In addition, several studies introduce the capability of estimating the total Above Ground Biomass (AGB) of individual plants from their spectral measurements ^{[131,132,141,146]}.

Development of automated plant detection methodologies represent a useful tool for estimating germination rates during early development stages, as well as determining the presence of weeds in rice fields, which can hinder the development of rice plants and reduce yields. To this end, the study ^[33] employs visible and NIR multispectral measurements, acquired using an UAV, to discriminate rice plants from background soil spectral measurements. Supplementally, an in-depth comparative study of the classification performance of various machine learning algorithms for discriminating between rice plants and common weeds is presented in ^[149]. ^[84] study the optimal height to operate a UAV to extract the Leaf area index LAI, and leaf dry weight (LDW) and evaluating the performance of hyperspectral images as a metric of how good their measurements were.

Currently, leaf color charts are distributed to rice farmers in order to estimate the N levels present in the plants. This visual cue is intended as a low-cost diagnostic tool in order to ensure adequate fertilization levels and to maximize yields ^[150]. However, previous work on the spectral properties of rice plants suggests that the 700-900 nm spectral region describes significantly superior statistical separability between spectral measurements of rice leaves at different fertilization levels ^{[119,120,126,130,151]}. Due to the fact that this optimal spectral range is outside the visible spectrum (380-740 nm) ^[136], hyperspectral imaging has been adopted as an imaging solution that is capable of visually capturing the differences in N levels in the appropriate spectral range. Thus, timely monitoring of N status of rice crops with remote sensing can aid the optimization of N fertilizer management and reduce environmental stresses for improving plant growth.

Spectral measurement sensors (either imaging or non-imaging spectrometers) have the sole function of acquiring the spectral response of the target of interest ^[70]. That is, these devices are not capable, nor is it their intended function, of directly measuring the physical magnitudes of relevance to rice crop health and productivity assessment efforts. For this reason, several studies have been performed towards building signal processing models to estimate rice plant and soil conditions of interest from their spectral responses. ^{[23,103,117,127,144,148,153]}.

4.2.   Chlorophyll and Nitrogen estimation

Table 3 summarizes the most relevant studies that use spectral measurements to estimate the N ^[31,118,120] or Chlorophyll ^{[137,144,152,153]} contents in rice plants using a variety of acquisition methodologies and processing techniques.

Of these physical magnitudes, Leaf Nitrogen Content (LNC) is the most common in the current state of the art, due to its closer relationship to rice crop yields. While the majority of these approaches attempt to directly estimate N levels from hyperspectral measurements, there have been efforts in further refining the accuracy of these models by incorporating additional sources of information, such as remote LIDAR measurements ^[74,117].

The LNC is an indicator of the N nutrients in the plants which an indirect variable extracted under different techniques from visible, Near infrared (NIR) or Short Wave Infrared (SWIR) data. It is used as indicator of the crops growth status, which it is related with the crop field. An estimation of the N content indicators in rice crop leaves using to different leaf clip meters is presented in ^[31], where two clippers provide an estimated value of N nutrient related indicators as: Chlorophyll, Flavonoid Content and Nitrogen Nutrient Index (NNI). By linear regression a comparative performance is presented for different fertilization rates and three experiments sets using various types of rice species. In ^[120], LNC is estimated from random leaf leave samples were collected from a single cultivar where different level of urea fertilizer is applied at seeding, tillering and shooting stages. Fluorescence intensity and ratio where performed at lab at 350 nm, 460 nm and 556 nm and compare the results and provides an insight of better performance of the regions where fluorescence data is used. Similar than ^[120], in ^[118] and estimation of the LNC from different samples collected at footing, heading and tillering stages. Additional their work propose a combine spectral index using hyperspectral LiDAR and fluorescence LiDAR. A study using SWIR to improve the accuracy in the estimation of LNC using Continuous Wavelet Analysis (CWA) is developed in ^[133], where they combine CWA and spectral index method using random data from a 4 set of experiments using rice and wheat cultivar to extract N_area, and N_mass with the leaf mass area (LMA). A difference of ^[133], in ^[120] use Back propagation Neural network BPNN combines with principal component analysis to show the spectral response of LNC. A single spectral study in ^[49] computes a spectral index and relates it to the N levels in canopy rice. In addition, it is of great interest to detect the presence of rice plants in the early season in order to reseed specific areas with convenient timing ^[33].

4.3.   Pest, diseases, heavy metal and pollutants

Regarding the different stages of plant growth, the state of the art on hyperspectral image analysis of rice plants at the seedling stage is focused on the detection of the presence of pests ^[35] and diseases ^[103,139] that can severely impact yields by preventing plants from developing into their productive stages. Conversely, studies on plants past the seedling stage are centered on the maximization of crop yield through the determination of various chemical properties indicative of plant health ^{[127,152,153]}. Due to the need to supply the growing world population with this basic crop, it is important to know the development cycles of this plant, as well as the factors, both climatic and environmental, that limit its development and yield, and the impact on human health. It is also necessary to consider all the methodologies and technologies available for crop analysis including hyperspectral images, canopy reflectance spectra, examine the mechanisms of pollutants uptake and translocation in rice ^[13], Phenology, remote sensing and near-surface imaging spectroscopy, among others. The presence of various contaminants and/or diseases can negatively impact the nutritional content of crops ^{[35,108,103,139,156]}. All the studies are summarized in Table 4 where diseases, pest, heavy metal contaminants and Pollutants are presented.

4.4.   Seed studies, rice lodging and other variables

Furthermore, hyperspectral imaging applications have been developed for specific aspects of the rice cultivation industry beyond plant health and growing status, such as identification of specific plant genotypes ^{[137,145,149,157]}, and seed type and vitality identification ^[91,125]. These studies address important concerns beyond the growing process and aid in the assessment of various aspects of grain quality, viability and/or geographical origins. These studies are summarized in Table 5.

5.   Signal processing methods

The mathematical relationship between the spectral measurements of rice plants extracted from hyperspectral images, and the actual physical magnitudes of interest in the plants to be estimated is highly nonlinear, and a nontrivial task to accurately determine ^[45]. When attempting to establish a mathematical model that describes the relationship between two such magnitudes, it is important to carefully evaluate the available established methodologies for achieving this task. The most relevant considerations in the surveyed literature regarding the specific methodologies used are summarized in Tables 6 and 7.

In this regard, a significant subset of the surveyed literature employs regression-based approaches to optimize for maximum correlation between specific spectral regions and the physical magnitudes of interest ^[23,139,148], where the proportion of the dependent variable's variance that can be explained from the selected independent variables ^[159] is taken as a performance measure (R² value). In these approaches, the spectral measurements (or a subset thereof) are approximated to be independent variables, or predictors, in a linear fashion for the specific dependent variables, or measured responses (such as N content, carbon content, etc.), being studied. Additionally, several studies employ correlation analysis between the behavior of a reduced subset of spectral bands and the target magnitudes of interest ^[49,127,157]. These approaches explore the correlation coefficient r, {r ∈ R|0 ≤ r ≤ 1} between individual spectral intensities x ∈ R, and the measured responses of interest y ∈ R:

where cov(x, y) is the covariance of the spectral measurement x and the magnitude of interest y, and s_x², s_y² are the variances of x and y, respectively. Several regression-based approaches perform a comparative statistical study to find a subset of spectral bands with high correlation to the variables of interest ^[31,132,152]. It is important to note that while this practice introduces an advantageous computational complexity reduction, it discards a significant number of spectral measurements that could be taken into consideration with the use of Principal Component Analysis (PCA) or similar dimensionality reduction techniques ^[119,134]. For classification-oriented applications, the surveyed literature presents a variety of methodologies including Support Vector Machines (SVM) ^{[50,125,126,149,159]}, Neural Networks ^[35] and Random Forest-based approaches ^[145].

Support Vector Machine (also, later generalized under the name Kernel Machine) classification of HSIs is a binary approach that relies on finding an optimal separating hyperplane by maximizing the perpendicular distance (given by support vectors) from the hyperplane to the closest data points from the two classes on both sides of it. In the case that a linear hyperplane is not able to satisfactorily discriminate between the two features with a significant margin using the dimensionality of the features, it is possible to map the features to be classified to a higher-dimensional space, in order to maximize class separability using a nonlinear kernel function (hence the alternate description of kernel machine) ^[160,161]. Compared to modern approaches (particularly, deep neural networks), SVM classification approaches consistently produce accurate results in applications with the limitation of reduced availability of training data ^[162].

Neural Network classification of HSIs is a non-parametric approach that employs the backpropagation algorithm ^[163] to train a neural network for pixel classification in a "black box" approach. In this case, a neural network is setup to have as many input nodes as spectral bands in a HSI pixel, one or more hidden layers with a variable number of nodes, and an output node consisting of the class assignment or inferred magnitude estimation for the input information. Alternatively, Convolutional Neural Networks (CNN) are Neural Network architectures optimized for operating on spatially distributed data, as is the case with HSIs ^[147,155], as opposed to Fully-Connected approaches that operate on a pixelwise manner ^{[118,120,164]}. While these approaches have recently introduced significant improvements on classification problems, they require vast amounts of training data to produce accurate models, which include nontrivial processing hardware requirements for the network training operation. These methodologies present the advantage of producing accurate statistical models for rice cultivation in reasonable processing times without the need of high-performance specialized hardware. However, the accuracy of the calculated models shows a consistent disadvantage when compared to alternate, more computationally complex, methodologies such as those based on PCA ^{[103,119,126,134,165]}, Support Vector Machines ^{[50,112,117,125,126,130,144,151]} and Artificial Neural Networks ^{[117,119,130]}. At the cost of increased accuracy, it is important to know that these methods describe a higher computational complexity, and it is important to take their required dedicated resource allocation and computing time when designing a deployable hyperspectral image analysis-based application.

5.1.   Classification-based hyperspectral imaging applications

In classification problems, where the employed methodologies produce discrete class outputs for detection purposes, results are stated using the Accuracy metric, calculated by dividing the number of accurately classified pixels from each respective study's testing set by the total number of pixels for each class ^[166]. As can be observed in several studies in Table 6 ^{[10,103,126,134,137,143,165]}, these discrete classification applications tend to produce results with high accuracy. This can be explained by the high statistical separability of the target spectra in these studies, which most commonly related to the presence various types of plant pests and/or diseases, when compared to the spectra of healthy plant tissue and background contents in specific wavelength ranges.

5.2.   Regression-based Hyperspectral Imaging Applications

Contrary to classification-based approaches, applications concerned with estimating a nondiscrete magnitude, such as LNC, from spectral measurements tend to describe lower performance values ^{[49,130,140,148,154]}. This discrepancy can be explained by the considerably larger continuous solution space of magnitude inference problems, compared to the discrete solution space in classification problems ^[167].

In order to improve the specific performance of the surveyed methodologies, several approaches incorporate an additional preprocessing step in order to maximize the statistical separability of key variables of interest in the acquired hyperspectral data. Depending on the required complexity of each methodology, this preprocessing step can range from a computationally inexpensive subset selection and/or scaling on the values of key spectral bands ^{[130,133,144,148]}, to a more computationally complex operation, such as Linear Discriminant Analysis PCA ^{[103,119,126,134,165]} that takes into account the behavior of the relevant spectral signatures across the available spectrum.

6.   Current challenges

We have observed a series of common challenges across the reviewed literature that represent limitations regarding the viability of deployment of hyperspectral imaging applications in the field. These challenges are summarized in Figure 6.

Due to the difficulty and cost of obtaining hyperspectral images of sites of interest, some of the surveyed studies perform field measurements at the locations of previously performed aerial hyperspectral imaging expeditions. This temporal mismatch between hyperspectral and field-collected measurements introduces a significant error into the final mathematical models due to a myriad of possible condition changes (illumination, change in vegetation composition, erosion, anthropogenic canopy changes, water content, etc.) between the acquisition of the input and target model data.

Because the acquisition and processing of these images into useful results for precision agriculture is an economically, computationally, and time intensive process, research efforts have been focused on finding static relationships between the acquired spectral measurements and the physical magnitudes of interest for a specific geography. Collectively, these approaches do not incorporate the modeling of the productivity influence of additional factors that are at risk of undergoing significant variations in behavior due to imminent climate change, such as shifts in temperature trends, soil and/or water acidity, wind and water current patterns, seasonal duration changes, among others. In addition, the surveyed literature is concerned with the study of the behavior of specific rice cultivars under the variation of specific relevant factors, such as fertilization levels. Thus, an important direction for future research is to begin working towards a unified characterization model for various cultivars and their requirements. As it is, the performance levels obtained in the surveyed literature do not necessarily imply that growing different cultivars under the same conditions would provide similar results. An accurate future productivity model that is able to adequately incorporate these factors would enable a significant increase in estimated yield predictions for any region around the globe.

Further, a study for hyperspectral images mountain forest canopy suggests that there is a nontrivial discrepancy in the measured spectral responses of remotely sensed canopies due to topographical irregularities ^[168]. While this is an issue that is less prevalent in most current rice fields due to their conventionally flat topographies, it is within the realm of possibility that planting areas will expand to less favorable topographies in the future due to the increasing need for additional farmland to support the growing population and compensate for the nutrient depletion of existing farmland.

Rice cultivation and, by extension, any hyperspectral imaging-based applications for its improvement, is a highly multivariate dynamic problem where a variety of intertwined factors such as soil composition, location, local macro/microclimates, fertilization levels, fertilizer usage, farming practices and a myriad of additional factors can impact the viability of a given method outside of the conditions present in individual research efforts. Given the imminent threat of global climate change which, beyond the effects of changes in climate, can have additional effects on rice cultivation practices, among which can be migration to new agricultural regions with more favorable climates but different soil chemical properties and the emergence of novel rice cultivars better suited to these conditions. In addition, even small supply chain disruptions can have a significant impact on the composition and efficacy of the various production inputs used in precision agriculture.

The surveyed literature consists mostly of studies performed over 1-5 growing seasons in a given region with known conditions. Currently, shifting climate dynamics around the world potentially threaten the long-term usability of the studies performed for any given region. Thus, hyperspectral imaging applications for rice cultivation will be required to take into account several dynamic factors and, most importantly, adequately model their influences on time-resolved recorded spectral measurements over multiple decades, whenever possible. It is our consideration that addressing these considerations will play a critical role in consolidating the long-term deployability and effectiveness of a given contribution in the face of imminent climate change.

Moreover, the lack of a standardized, publicly-available dataset for the development of signal processing methodologies for various aspects of the rice cultivation using hyperspectral images requires individual research groups to plan and execute their data collection expeditions over significant stretches of time, spanning up to various cultivation cycles ^[23]. Beyond this, it hinders the capability of research groups in different geographies to accurately compare their results to the available literature by isolating the differences in spectral responses between local cultivars and soil compositions. In the surveyed literature presented in Tables 2 to 5, it is apparent that a significant majority of studies relies on manual acquisition of data using specialized spectral measurement equipment, such as spectrometers and/or hyperspectral cameras. These approaches require, in addition to the nontrivial matter of the equipment acquisition, skilled personnel capable of operating the equipment in a manner conductive to estimating the relevant magnitudes for a given application. These concerns present an important limitation on the widespread deployment of laboratory-derived innovation to applications in the field. In contrast, a significant minority of the surveyed literature describes systems capable of automated acquisition using UAVs ^{[33,74,135,138,146,147,148,153]}. In order to provide better accessibility of the technological advances being developed, it is our consideration that it is necessary for either public or private actors to further advance the consolidation and deployability of these analytical studies in order to facilitate their availability as a service, either directly to crop farmers in order to maximize the efficiency of their efforts, or towards funding agencies in order to streamline the process of evaluating the development of their investments, and better calibrate their investment return expectations to increase availability of funding sources for crop producers. One major limitation consistent across the majority of surveyed literature is the use of manual measurements to compute the published models. In order to minimize potential ongoing costs for widespread hyperspectral technology deployments to rice growers, it is important to continue producing models based on UAV-acquired hyperspectral data. These approaches enable the acquisition of a significantly higher volume of data, in an easily scalable manner according to plantation size with reduced labor requirements. On the topic of widespread deployability to growers, it is our consideration that additional research effort is required on the development of solutions that provide succinct, actionable insights to end users to maximize the effects of their labor. An example of such applications is the work published in ^[24], which uses the inferred LNC to provide end users with an indication of the required fertilizer levels to be applied. Beyond the possibility of providing instructions to a human operator, further work on these applications can play a pivotal role in the context of a precision agriculture system that distributes fertilizers through automated actuators with little to no human intervention, further reducing the potential impact on food production due to population migration and worker shortages.

It is important to highlight that most previous studies include the cultivation of rice plants under predefined conditions in a controlled environment in order to isolate the relevant parameters for each study. Thus, in order to successfully perform a large-scale deployment of the findings of the surveyed literature, it is required to dedicate future research efforts in the modeling of the influence of various local conditions on the health and productivity of rice crops outside of the specific regions of study. Success in these efforts would enable efficient deployments based on these findings in a wider variety of regions with different soil compositions and climatological conditions. In addition, in the current atmospheric conditions, it is becoming increasingly necessary to model the influence of atmospheric pollution both on the health of rice plants in existing plantations, and on the viability of soil for the informed allocation of land for new plantations in order to satisfy growing production needs.

7.   Conclusions

The current review synthesizes and highlights the most relevant works regarding the use of hyperspectral imaging sensing technology and hyperspectral image processing algorithms for the estimation and monitoring of various aspects of the rice cultivation process. The significant contributions in literature are summarized according to their objectives and summarized in tables according to the specific constraining factors considered such as biomass and vegetation estimation, landcover classification of rice growing sites; Chlorophyll and N content estimation; Studies of diseases, pests, heavy metals and other pollutants' effects on rice crops; Studies focused in various seed properties, lodging and others variables. Relevant information about the studies performed are presented, such as the acquisition methodology, considered spectral region, rice cultivars used in each work and the region of the study, which can be used as a reference of local climate conditions.

The hyperspectral signal processing methods used in rice production are summarized in terms of whether the objective of the individual studies is to infer a specific magnitude, or to classify the spectral measurements into discrete classes. Further, the specific approach used in each study, where the most relevant methodologies are discussed, such as statistical, linear or multiple spectral transformations and combined advanced methodologies. Additionally, the current review identifies the common challenges present across the proposed hyperspectral systems for rice cultivation that, in our consideration, are highly relevant to the goal of developing widely deployable applications of this technology, with the objective of supporting the efficient agro-industrial production of rice crops. The widespread deployment of the surveyed technological advancements is crucial for achieving global sustainability requirements, which is an important challenge in the current age. Further, these developments enable the advancement of various important global challenges related to uplifting communities from poverty, and to provide them with equal opportunities in a manner that is environmentally sustainable.

Regional or country-scale remote sensing deployments of rice field mapping and yield-estimating technologies can be a valuable planning tool for policymakers and help to reach established Sustainable Development Goals. Successful integration of these methodologies into the decision-making process can significantly reduce potential food waste and economic losses through more accurate import/export rates. For these reasons, it is of great importance for upcoming research to focus on the development of low-cost, user-friendly spectral imaging devices that can be more feasibly deployed in growing operations of varying scales.

Finally, in order to more effectively extract as much information from the necessary higher volume of time-resolved hyperspectral remote sensing images, and adequately model the influence and trends of shifting climate conditions and its effects on rice cultivation in a precision agriculture setting, it is imperative to take advantage of any developments in multidimensional signal processing methodologies for feature extraction applications.

Acknowledgments

This research was funded by Grant/Award Number: FID 2018-096 of the Research funding programs of the National Secretariat for Science, Technology, and Innovation of Panama (SENACYT). The authors acknowledge also, SENACYT for the support through its Panama National Research System (SNI by his acronym in Spanish).

Conflict of interest

The authors declare no conflict of interest. The funding entity had no role in the design of the study; in the collection, analyses methodologies, or interpretation of data; in the writing of this manuscript, or the decision to publish the results.