Discriminator-free adversarial domain adaptation with information balance

Abbreviations: UDA:Unsupervised Domain Adaptation; MMD: Maximum Mean Discrepancy; ND: Normalized Distance; DBDA: Dynamic Balance-based Domain Adaptation; GANs: Generative Adversarial Networks; MCD: Maximum Classifier Discrepancy; CGDM: Cross-Gradient Difference Minimization; GRL: Gradient Reversal Layer; LDA: Linear Discriminant Analysis.

1.   Introduction

In recent years, multitude methods in computer vision tasks have been explored based on deep neural networks. Nevertheless, traditional deep learning-based image classification techniques necessitate a vast amount of labeled data for training. Acquiring these data and annotations is both time-consuming and costly, thus making it impractical for various scenarios ^[1,2]. Additionally, the effectiveness of most classification models relies on the assumption that the distributions of the training data (source domain) and test data (target domain) are either identical or similar, terefore leading to a significant drop in performance when the model is tested.

To address this challenge, research teams have delved into Unsupervised Domain Adaptation (UDA)^[3,4]. UDA aims to extract domain-invariant features from both the source and target domains, thereby transferring knowledge from the labeled source domain to the unlabeled target domain, which experiences a domain shift, thereby training a model with robust generalization capabilities^[2,5,6].

Initially proposed by many researchers, UDA methods should explore representations that learn domain-invariant knowledge^[7,8]. In recent years, several different approaches have emerged based on this idea, with some widely recognized directions including the following: domain discrepancy measurement-based methods^[5,9], contrastive learning-based methods^[10,11], and adversarial learning-based methods^[12,13], all of which have achieved notable success. Domain discrepancy measurement-based methods aim to minimize discrepancy metrics, such as the Maximum Mean Discrepancy (MMD) method. Contrastive learning-based methods align feature representations by comparing the representation vectors of samples from the source and target domains, thereby using a contrastive loss to bring positive sample pairs closer and to push negative sample pairs apart. Adversarial learning-based methods typically use adversarial training between generators and discriminators to implicitly derive the domain-invariant features^{[13,14,15,16,17]}.

Adversarial Unsupervised Domain Adaptation (UDA) methods leverage an automatic adversarial mechanism between a generator and a discriminator to effectively minimize inter-domain discrepancies. These approaches not only reduce the interference of manually designed factors, but also flexibly adapt to various neural network architectures and tasks^[12,18,19]. Building on these advantages, we conducted a systematic study of existing adversarial UDA methods to explore their performance in real-world tasks and to identify potential areas for improvement.

In our study, we identified two primary design paradigms in task-oriented adversarial training. The first involves establishing two independent discriminators, where each discriminator adversarially interacts with the generator on the same image sample, thereby focusing on category-level and domain-level alignment, respectively. By comparing the outputs of the two discriminators, this approach can partially mitigate the class-level domain discrepancies during the transfer process. However, it is susceptible to blurred predictions, which complicate the generator's optimization and may lead to local optima during training.

Alternatively, using a single discriminator to classify the domain of the sample primarily emphasizes domain-level feature alignment. While this method performs well in capturing the overall domain characteristics, it is less sensitive to class-level information, which may degrade the category differentiation. Over a prolonged training, the generator is prone to issues such as mode collapse, thus ultimately resulting in suboptimal transfer performance^[8,20].

Moreover, we observed that existing methods often overlook the issue of a sample imbalance between the source and target domains. During training, the domain with a larger number of samples tends to dominate, thus introducing a bias into the model. This bias can weaken the ability to achieve a domain alignment and may even lead to a negative transfer, therefore significantly affecting the model's performance and generalization capabilities.

To address the issues present in adversarial methods, we explored a different approach by directly reusing the classifier as a discriminator. Simultaneously, we introduced a new domain bias quantification method, called te Normalized Distance (ND), thus enabling the classifier to focus on both the domain-level and the class-level information.

To further address the issue of a domain sample imbalance, we introduced a dynamic balancing learning method that adjusts the sample weighting strategy based on differences in a sample quantities and variations in the class discriminability. Our approach updates the weight allocation in real time during training, thus enabling the model to progressively achieve a balanced attention to both the source and target domain samples over an iterative training. This prevents the bias or negative transfer issues caused by imbalanced data distributions.

Based on the aforementioned exploration, we propose a Dynamic Balance-based Domain Adaptation (DBDA) method for the self-correlated domain adaptive image classification. Our model achieves adversarial UDA training by reusing the classifier.

This algorithm dynamically adjusts the training weights based on the alignment status between the source and target domains, thus effectively handling the distribution discrepancies and the sample imbalance.Our method aligns the domain-level information while preserving the key class discriminative features, thus preventing information loss during feature transfer. The classifier reuse strategy reduces the complexity of the training process, accelerates the model convergence, and significantly enhances the model's ability to jointly represent the domain and class features in a simple yet effective manner.

Additionally, our method employs a dynamic balancing algorithm to achieve a balanced state between the domain alignment and the class alignment, thus ensuring a high accuracy for both. During the transfer process, our method effectively balances the attention between the inter-domain distribution differences and the class discriminative ability. As a result, it trains a high-performance classifier with a strong generalization capability, thus providing a simple yet efficient solution for domain adaptation tasks. We conducted extensive experiments on multiple datasets, and the results demonstrate that the proposed method outperforms several state-of-the-art approaches.

The main contributions of our research are summarized as follows:

● We propose an adversarial training method that reuses the classifier as the discriminator, thus eliminating the need for additional discriminators and significantly reducing the training complexity and computational overhead. By reusing the classifier, the model can efficiently leverage the classifier's prediction information, thereby reducing information redundancy and potential biases during the domain alignment process.

● Our approach demonstrates an excellent performance in aligning intra-class and inter-class correlations in the target domain through classifier reuse. This leads to the distribution of target domain samples that gradually approach that of the source domain. At the same time, it efficiently captures the class discriminative features, thus achieving the joint alignment and optimization of both the domain-level and class-level features.

● We introduce a dynamic balancing mechanism that adjusts the model's weight distribution between the classification and adversarial tasks in real-time based on the alignment progress during training in both the source and target domains. This strategy effectively balances the requirements of the domain alignment and the class alignment, thus preventing situations where one task achieves a high accuracy while the other underperforms, thereby further enhancing the model's overall performance and transfer robustness. The code and datasets are available at https://github.com/Jwriter-2000/Domain-Adaptation-Dataset.

2.   Related work

2.1.   Unsupervised domain adaptation

As a branch of transfer learning, UDA has seen rapid development in recent years^[1,2]. UDA aims to leverage partially annotated source domain data and unlabeled target domain data to build a model with a generalization capability^[4,5,21]. Existing methods broadly fall into several directions: domain discrepancy measurement-based learning methods (e.g., Maximum Mean Discrepancy)^[5], contrastive learning-based methods^[10,11], and adversarial-based methods^[13,18]. These approaches aim to obtain feature representations with domain-invariant properties to reduce cross-domain distribution shifts.

Methods based on difference measurements primarily align two domains by quantifying the domain discrepancies, often employing MMD as a metric. Long et al. proposed minimizing the multi-kernel MMD between two domains along with classification prediction errors, thus learning abstract representations of features at different levels to align the domains^[21]. Meanwhile, Ren et al. designed a Conditional Kernel Bures distance based on conditional distribution differences to offer an interpretable transfer method^[22].

Contrastive methods create positive and negative sample pairs to construct contrastive loss, thereby aiming to bring similar samples from different domains closer for feature alignment^[10]. Research by Shen et al. demonstrated that contrastive learning effectively decomposed information at both domain and class levels, thus enabling the transfer of knowledge from source to target domains even with significant domain differences^[23]. The method proposed by CDA integrated both contrastive loss and MMD to narrow the gap between the source and target domains, thus achieving notable accuracy improvements^[24]. Furthermore, methods based on disentangled representation learning decompose complex data representations into independent and meaningful latent factors ^{[25,26,27,28,29]}. Their strong robustness and transferability provide significant insights for research on UDA tasks.

Adversarial methods, inspired by Generative Adversarial Networks (GANs), aim to minimize the difference between the source and target domains through adversarial games^[30]. These methods typically consist of a generative model $G$, which learns to capture the data distribution, and an additional discriminator model $D$, which is tasked with distinguishing whether samples belong to the source domain or the target domain. Ganin et al. first introduced adversarial methods to domain adaptation tasks with the Domain Adversarial Neural Network (DANN), thus showcasing GAN's strong transfer capabilities in UDA tasks^[12]. Subsequent research, as proposed by Long et al., integrated learned features with discriminator predictions to align features across different domains^[31].

In addition to methods that used an additional discriminator, some teams have explored integrating two task-specific classifiers as discriminators, thereby using the difference in their outputs to guide adversarial training. Maximum Classifier Discrepancy (MCD) initially used the L1 distance to represent discrepancies between two classifiers' outputs^[5]. Meanwhile, other teams have investigated various methods to measure discrepancies, thereby proposing a series of valuable approaches. Cross-Gradient Difference Minimization (CGDM) adopts a cross-domain gradient difference to alleviate the domain discrepancies^[9].

Besides the aforementioned methods, knowledge distillation based on teacher-student models is also one of the research hotspots. For example, ^[32] proposed a multi-view latent space learning framework that achieved reliable pseudo-labeling through a multi-view contrast. ^[33] fused the similarity relationships predicted by different teacher networks as supervision to optimize the student network, which involved more sample relationships, ultimately achieving domain-invariant knowledge transfer. Meanwhile, graph learning-based methods ^[34] also show considerable potential in UDA tasks.

The first type of adversarial training method typically relies on additional discriminators, which not only increases the complexity and computational cost of model training, but may also lead to biases in the discriminator when the data distributions between the source and target domains are inconsistent, thereby affecting the effectiveness of the domain alignment. On the other hand, methods without specific discriminators, while simplifying the structure, often focus only on within-class differences and lack a global control over the inter-class and inter-domain differences, thus making it easier to fall into suboptimal solutions. Moreover, these methods often excessively pursue a complete alignment of source and target domain features, thus neglecting the potential distributional differences and sample imbalance between the two. This oversight can lead to a domain misalignment during the domain adaptation process, thus reducing the model's transferability and classification performance. To address these issues, our model proposes a classifier-reuse adversarial method, thus establishing a new dynamic balance discrepancy measurement algorithm. This approach considers both domain and class information, thereby dynamically weighting them based on an alignment during training, which ultimately results in a well-performing classifier.

2.2.   Discriminator-free adversarial learning and dynamic balancing

Unlike traditional GAN models, methods based on Discriminator-free approaches do not require the explicit design or training of a discriminator to distinguish between generated data and real data. These methods reduce the complexity of the model training process by eliminating the discriminator and avoid common issues in the GAN training, such as the imbalance between the discriminator and the generator. Singh1^[35] proposed a method that fit a two-component Gaussian mixture model for source and target predictions, where the resulting Gaussian distribution was used to define an adversarial loss based on Frechet distance. Chen^[36] demonstrated that Discriminator-free methods are also effective in well-aligned multimodal images. Discriminator-free Adversarial Learning is an important research direction in adversarial domains, especially with the potential for improved efficiency and stability.

Dynamic Balancing adjusts the system parameters in real time based on the training progress or certain specific metrics, thus enabling the model to balance the relationships between different tasks, objectives, or data during training. Many existing studies have incorporated Dynamic Balancing into UDA tasks from various perspectives, such as ^[38]. In adversarial training, balancing the training of the generator and discriminator is crucial, and dynamic balancing methods can adjust the training strategies between the generator and discriminator to ensure the overall stability of the model. Our method updates the weight distribution in real time during training, thus enabling the model to progressively focus on balancing the source and target domain samples during iterative training, thus avoiding issues such as domain shift or negative transfer caused by data distribution imbalance.

3.   Proposed method

Our method's overall framework is illustrated in Figure 1. We use a pre-trained source domain model to initialize the training process.

Figure 1.  The overall framework is divided into three parts: the sample weighting module, the classification adversarial module, and the dynamic balancing module. First, we perform sample weighting by combining the source and target domain data. Then, the domain adaptation process is conducted, where a single classifier handles both the classification and discrimination tasks. During the forward pass, the class alignment task is performed, and during backpropagation, the parameters are updated through a gradient reversal layer, thus enabling adversarial training between the feature extractor $G$ and the classifier $C$. Finally, dynamic balancing is employed to control the class alignment and the domain alignment throughout the process.

DownLoad: Full-Size Img PowerPoint

To address the issue of differing sample sizes between the two domains, we first weight the samples according to their quantities to reduce errors caused by an imbalanced spatial distribution. Then, the weighted data is mapped into the feature generator $G$ and the classifier $C$, which is specifically designed for the classification task, thus generating the corresponding predictions. Subsequently, the classifier is combined with the Normalized Distance (ND) and used as a discriminator. The model consists of a pre-trained ResNet-based generator $G$ and a classifier $C$ constructed from fully connected layers and a softmax layer. During the forward propagation, the Gradient Reversal Layer (GRL) remains inactive. However, during backpropagation, the GRL multiplies the gradient by a negative scalar to reverse the gradient, thus maximizing the domain discrimination loss. This process enables the learning of the domain-invariant features and completes the transfer classification task, thus allowing the module to automatically update its parameters. Finally, we developed a dynamic balancing parameter that adjusts the weights of the domain alignment loss and class discriminability loss, thus achieving a dynamic balance between the domain alignment and the class alignment. This ensures that both aspects are equally prioritized and aligned at multiple levels.

3.1.   Sample reweighting

First, let's define that we have a source domain containing ${N_S}$ labeled data samples ${D_S} = \left\{ {\left({x_i^s, y_i^s} \right)} \right\}_{i = 1}^{{N_S}}$, where $x_i^s \in {X_s}$, $y_i^s \in {Y_s}$, with labels covering $k$ categories. Additionally, we have an unlabeled target domain sample ${D_{\rm{t}}} = \left\{ {\left({x_t^i} \right)} \right\}_{i = 1}^{{N_t}}$, where $x_i^t \in {X_t}$. During training, when the number of image samples in one domain significantly exceeds that in the other, the imbalance can lead to a higher attention and bias in the model, thus resulting in negative transfer issues. To address this, we reweight the two domains based on sample quantities to balance the weights. The reweighting method is as follows:

where ${n_s}$ and ${n_t}$ represent the number of samples in the source and target domains, respectively, and $\lambda \in (0, 1]$ is a hyperparameter used to dynamically control the sample weights. After reweighting, the samples from both domains are fed into the feature generator for the next stage of training.

3.2.   Adversarial learning strategy without specific discriminators

Inspired by GANs, adversarial learning-based UDA methods typically require an additional discriminator $G(\cdot)$ and a generator $D(\cdot)$ for adversarial training. The generator extracts features $f^s = G(x^s)$ and $f^t = G(x^t)$ from the samples, and the discriminator $D(\cdot)$ outputs the corresponding predictions. Then, the classifier $C\cdot)$ outputs the predicted probabilities. The loss functions are usually defined as follows:

where ${{\cal L}_{{\rm{ce}}}}$ is the classification loss, and ${\tilde {\cal D}_s}$ and ${\tilde {\cal D}_t}$ represent the distribution of generated features for ${{\cal D}_{\cal S}}$ and ${{\cal D}_{\cal S}}$, respectively. Common adversarial learning-based UDA methods can be categorized into two types based on whether an additional discriminator is used. Since the original task-specific classifier can implicitly discriminate between the source and target domains, we opt to reuse the classifier $C$ as the discriminator $D$.

We start by training on labeled samples from the source domain, where supervised training significantly enhances the classification accuracy, thus enabling the classifier to provide correct answers during the prediction. Conversely, during unsupervised training on the unlabeled target domain, the classifier tends to produce incorrect predictions. We construct a self-correlation matrix that includes true labels, predicted labels, and a prediction accuracy. This matrix serves to reflect the intra-class and inter-class correlations in the data. During training, we utilize the differences in the self-correlation matrices between the source and target domains to construct an evaluator to optimize adversarial domain adaptation methods. It is observed that for a dataset with $h$ samples and $k$ classes, the classifier $C$'s prediction matrix $Z = C\big(f\big)\in\mathbb{S}^{h\times k}$ for the samples can be expressed as follows:

Therefore, the self-correlation association matrix is defined as $R = Z^TZ\in\mathbb{S}^{k\times k}$. For the self-correlation matrix $R$, its main diagonal elements are used to compute the intra-class correlations, while off-diagonal elements are used to compute the inter-class correlations. To achieve domain alignment, we attempt to bring the differences in the target domain closer to those in the source domain. We define the overall intra-class correlation as $I_{a}$ and inter-class correlation as $I_{b}$:

For the source domain, predictions tend to increase the value of $I_{a}$ and decrease the value of $I_{b}$. Conversely, in the target domain, due to the lack of supervised training, $I_{a}$ tends to decrease while $I_{b}$ increases. Therefore, the domain differences can be represented by $L_a-L_e$. According to equation (3.4), the domain differences are defined as $L_{a}+L_{e} = h$, where $L_{a}$ are the Frobenius norms of the prediction matrices for the source and target domains, respectively. Thus, we have $\Delta M = \| P_s \|_F - \| P_t \|_F$. Since $Z$ is predicted by the classifier $C$, $2\|C\|_F-h$ can be used as the correlation assessment function. Furthermore, considering that the weights and samples size $h$ are constants, we can directly use $\left\|C\right\|_F$ as the critical function for the correlation assessment.

To attempt to balance the importance of the source and target domain data, we introduce a K-Lipschitz evaluation function $H$. This function aims to assign high scores when evaluating the representations of the source data $f\in D_{s}$ and low scores when evaluating the representations of the target data $f\in D_{t}$, while also considering the 1-Wasserstein distance $W_1(\tilde{D}_s, \tilde{D}_t)$ between the two feature distributions $\tilde{D}_s, \tilde{D}_t$:

where $\left\|\bullet\right\|_{L}$ denotes the Lipschitz semi-norm [2], and $P$ represents the Lipschitz constant. However, as mentioned above, $\left\|C\right\|_{F}$ can effectively serve as a critic function for the discriminator $D$. This function can be expressed as follows:

Therefore, by employing this approach, the domain alignment and class distinction can be simultaneously achieved under a unified objective. This method enables a better utilization of the discriminative information from predictions to capture the multimodal structures within the feature distribution.

However, adversarial learning based on the Frobenius norm 1-Wasserstein distance tends to shift categories with a small number of samples towards adjacent categories with a large number of samples, rather than aligning them along the decision boundary. This results in a lack of diversity in the predictions. Inspired by the nuclear norm, maximizing the nuclear norm ${\left\| Z \right\|_*}$, which implies maximizing the rank of $Z$ when the Lipschitz semi-norm ${\left\| {\rm{ \bullet }} \right\|_L}$ is nearby, maintains the diversity of predictions. Therefore, we use the nuclear norm instead of the Frobenius norm:

The discriminator can be expressed as $D = \|C\|_*$. In this model, there is no separate discriminator; the classifier performs the task instead. This helps achieve a feature-level alignment when the classifier is used for classification, thus facilitating a class-level alignment. Since the classifier acts as an implicit discriminator and its components satisfy the P-Lipschitz constraint, additional weight clipping and gradient penalty strategies are not needed for training. Consequently, the maximized domain critic loss can replace the nuclear-norm discrepancy:

In this adversarial learning framework, to enhance the diversity of the model's alternating updates, we use a gradient reversal layer (GRL) that does not include the aforementioned weight clipping and gradient penalty strategies. Under these conditions, the model training follows a max-min approach:

Meanwhile, to prevent the source domain data from excessively influencing the prediction results and to ensure the authenticity of the UDA classification, both the generator $G$ and the classifier $C$ need to be optimized by minimizing the classification loss :

3.3.   Dynamic balancing strategy

To ensure a good balance between the alignment and classification, and to have both working together towards better results, this section introduces a dynamic balancing factor. This factor controls the domain alignment loss and class separability loss, thus allowing for the real-time monitoring of the alignment and separability during the training process. We use the Maximum Mean Discrepancy (MMD) and Linear Discriminant Analysis (LDA) to measure the current feature representation's cross-domain alignment and separability, respectively. MMD is primarily used to measure the distance between two different but related distributions:

The LDA-based separability estimator $\max L\left(D \right)$ is given by the following:

where ${P_b}$ represents the between-class scatter matrix, and ${P_w}$ denotes the within-class scatter matrix. Clearly, the larger the value of $L\left(D \right)$, the better the discriminative performance of the method.

Since the evaluation criteria of the two measures are not on the same scale, their estimated values are normalized using min-max scaling to linearly transform the evaluation values and map the results to the range [0, 1]. This establishes a dynamic balancing factor. Let $M\tilde MD\left({{D_{\rm{s}}}, {D_t}} \right)$ and $\tilde L\left(D \right)$ represent the normalized values of $MMD\left({{D_{\rm{s}}}, {D_t}} \right)$ and $\max L\left(D \right)$, respectively:

Through normalization, the equations are $M\tilde MD\left({{D_s}, {D_t}} \right) \in (0, 1]$ and $\tilde L\left(D \right) \in (0, 1]$. Thus, a dynamic balancing factor is constructed as follows:

In this equation, a smaller $M\tilde MD\left({{D_s}, {D_t}} \right)$ indicates a better domain alignment, and a smaller $1 - \tilde L\left(D \right)$ indicates a better class separability. When $M\tilde MD\left({{D_s}, {D_t}} \right)$ approaches 0 and $1 - \tilde L\left(D \right)$ approaches 1, it signifies that the alignment effect is much better than the classification effect, thus resulting in $\alpha$ approaching 0. Conversely, when $M\tilde MD\left({{D_s}, {D_t}} \right)$ approaches 1 and $1 - \tilde L\left(D \right)$ approaches 0, it indicates that the classification effect is much better than the alignment effect, thus resulting in $\alpha$ approaching 1. Based on this, $\alpha$ is used as the weight for the domain alignment loss, and $1 - \alpha$ is used as the weight for the class separability loss. When $\alpha$ and $1 - \alpha$ are close, it indicates that the alignment effect and the classification effect are also close. Therefore, the dynamic weighting model for domain alignment and class discrimination is as follows:

3.4.   Overall loss function

In summary, our method ultimately aims to optimize an overall loss function:

where the dynamic balancing factor $L_{adv}$ is used to weigh the domain alignment loss, and $L_w$ is used to weigh the class separability loss, thus ensuring that both the alignment and classification effects are balanced throughout the training process.

4.   Experiments

In this section, we will introduce the dataset and model parameter settings, give the experimental results of our proposed method, compare it with other methods, and finally analyze and interpret the experiments from multiple perspectives of data and visualization.

4.1.   Datasets and experiments settings

We performed experiments on four classical UDA image classification datasets, namely Office-31^[39], ImageCLEF^[31], Office Home^[4], and VisDA-2017^[40]:

(1) Office-31 is a common benchmark dataset for domain adaptation, covering images from three domains: Amazon products (A), DSLR cameras (D), and webcams (W). Each domain is comprised of 31 categories with a total of 4110 images. We experimented with all domain combinations.

(2) Image-CLEF is also a widely used standard domain adaptation dataset, consisting of three domains with significant style differences: Caltech-256 (C), ImageNet ILSVRC2012 (I), and PASCALVOC2012 (P). Each domain contains 600 images from 50 categories. We conducted all six experiments on the Image-CLEF dataset.

(3) Office-Home is a highly challenging UDA dataset, containing samples from four distinct domains: art images(A), clip art(C), product images(P), and real-world images(R). It includes 15,500 images of 65 categories of everyday objects in office settings.

(4) VisDA is a large-scale domain adaptation dataset comprised of two domains: synthetics and real, with over 280k images across 12 categories.

According to the specifications of unsupervised domain adaptation, labeled source domain samples and all unlabeled target domain samples are involved during the training process. The proposed method is implemented based on the PyTorch framework. We use the SGD optimizer for training with a momentum of 0.9, a weight decay of 1e-3, a batch size of 36, and a cropped image size of 224 × 224 for the model. The initial learning rate for the classifier C is set to 5e-3, which is 10 times that of the feature extractor G.

Baseline: To evaluate the effectiveness of our method, we choose a baseline that does not reuse a discriminator, using MMD loss as the adversarial loss, and setting the balance parameters $\alpha$ and $(1-\alpha)$ to 0.5 and 0.5 for the transfer model. We compare this baseline with our method on the aforementioned dataset to validate the effectiveness of our approach.

4.2.   Comparison with the state-of-the-art

The results for Office-Home are shown in Table 1. By comparison, it is evident that the proposed method significantly improves the classification accuracy. For instance, in tasks with substantial inter-domain differences and class imbalances, such as A→R and C→R, our method achieves accuracies of 80.1 and 74.8%, respectively. Furthermore, in the P→A, P→C, and P→R tasks, the obtained accuracies are consistently higher than those of the existing methods on average. This improvement is attributed to the introduction of constraints, the effective utilization of predicted discriminative information, and maintaining a balance between the domain alignment and the class alignment.

Table 1.  Results on Office-home. (ResNet-50).

Methods	A$\to$C	A$\to$P	A$\to$R	C$\to$A	C$\to$P	C$\to$R	P$\to$A	P$\to$C	P$\to$R	R$\to$A	R$\to$C	R$\to$P	Avg
ResNet-50[41]	34.9	50.0	58.0	37.4	41.9	46.2	38.5	31.2	60.4	53.9	41.2	59.9	46.1
WDGRL[42]	44.1	63.8	74.0	47.3	57.1	61.7	51.8	39.1	72.1	64.9	45.9	76.5	58.2
JAN[31]	45.9	61.2	68.9	50.4	59.7	61.0	45.8	43.4	70.3	63.9	52.4	76.8	58.3
DANN[2]	45.6	59.3	70.1	47.0	58.5	60.9	46.1	43.7	68.5	63.2	51.8	76.8	57.6
MCD[5]	48.9	68.3	74.6	61.3	67.6	68.8	57.0	47.1	75.1	69.1	52.2	79.6	64.1
HAFN[43]	50.2	70.1	76.6	61.1	68.0	70.7	59.5	48.4	77.3	69.4	53.0	80.2	65.4
ETD[44]	51.3	71.9	85.7	57.6	69.2	73.7	57.8	51.2	79.3	70.2	57.5	82.1	67.3
FGDA[45]	51.8	72.0	79.2	62.7	72.0	73.9	60.0	49.7	79.2	69.5	56.8	82.3	67.4
DFE-DA[46]	56.4	74.9	78.2	62.8	72.3	73.2	62.3	53.8	80.4	72.4	60.5	82.8	69.2
ETA[44]	51.3	71.9	85.7	57.6	69.2	73.7	57.8	51.2	79.3	70.2	57.5	82.1	67.3
DMDA[47]	55.7	75.9	78.6	56.7	76.1	73.9	57.9	51.2	79.2	66.8	57.6	82.6	67.7
BuresNet[42]	54.7	74.4	77.1	63.7	72.2	71.8	64.1	51.7	78.4	73.1	58.0	82.4	68.5
AEGDM[48]	56.0	70.7	79.8	64.6	73.1	71.6	61.2	53.1	79.1	71.6	59.7	84.3	68.8
Baseline	55.3	70.9	76.2	62.1	69.8	75.6	60.8	53.2	78.4	69.3	58.6	80.7	67.6
Ours	58.7	75.6	79.5	64.8	75.3	76.7	65.5	56.7	81.2	74.6	59.8	86.0	71.2

---

 | Show Table

DownLoad: CSV

The results for ImageCLEF are shown in Table 2, which presents the experimental outcomes of six domain adaptation tasks and the average accuracy on this dataset. Our method achieves the optimal average accuracy of 90.4%, with the best performance in the I→P and P→I tasks.

Table 2.  Results on Image-clef. (ResNet-50).

Methods	I$\to$P	P$\to$I	I$\to$C	C$\to$I	C$\to$P	P$\to$C	Avg
ResNet-50[41]	74.8	83.9	91.5	78.0	65.5	91.2	80.7
WDGRL[42]	76.8	87.0	91.7	87.2	75.2	90.3	84.7
MCD[5]	77.3	89.2	92.7	88.2	71.0	92.3	85.1
JAN[31]	76.8	88.0	94.7	89.5	74.2	91.7	85.8
DANN[2]	75.0	86.0	96.2	84.0	74.3	91.5	85.0
CAT[49]	76.7	89.0	94.5	89.8	74.0	93.7	86.3
AEGDM[48]	81.4	93.2	98.1	92.1	78.1	96.5	89.9
IWCA[50]	77.5	91.3	97.0	90.5	75.8	95.3	87.9
MEDM-LS[51]	78.2	93.3	97.2	93.0	78.3	95.5	89.3
CGDM[9]	78.7	93.3	97.5	92.7	79.2	95.7	89.5
Baseline	80.6	89.4	94.9	89.3	75.4	92.6	87.0
Ours	83.0	94.5	98.0	92.5	78.0	96.3	90.4

---

 | Show Table

DownLoad: CSV

The results for Office-31 are shown in Table 3. The proposed model achieves an optimal performance with an accuracy of 88.9% on average. Specifically, in the task A→W, our method achieves a high accuracy of 93%, which is the highest among the comparative models. In the relatively more challenging tasks of D→A and W→A, our method demonstrates a relatively stable performance despite the significant domain distribution inconsistency and sample imbalance issues such as adapting from domains with fewer samples (D and W) to a domain with more samples (A). This indicates that our model not only has a strong robustness in the domain alignment, but is also capable of effectively handling the complexity of the class distribution caused by a larger number of target domain samples.

Table 3.  Results on Office-31. (ResNet-50).

Methods	A$\to$W	D$\to$W	W$\to$D	A$\to$D	D$\to$A	W$\to$A	Avg
ResNet-50[41]	68.4	96.7	99.3	68.9	62.5	60.7	76.1
DAN[21]	80.5	97.1	99.6	78.6	63.6	62.8	80.4
WDGRL[42]	72.6	97.1	99.2	79.5	63.7	59.5	78.6
DANN[2]	82.6	96.9	99.3	81.5	68.4	67.5	82.7
ADDA[52]	86.2	96.2	98.4	77.8	69.5	68.9	82.9
MADA[53]	90.0	97.4	99.6	87.8	70.3	66.4	85.2
CAT[49]	91.1	98.6	99.6	90.6	70.4	66.5	86.1
ETD[44]	92.1	100.0	100.0	88.0	71.0	67.8	86.2
JBL[54]	91.2	97.6	100.0	86.9	70.5	71.8	86.3
DFE-DA[46]	88.3	99.4	100.0	87.6	74.3	73.1	86.9
DMDA[47]	91.6	98.6	99.4	90.0	73.8	74.0	87.7
BSWD[55]	90.1	99.0	100.0	89.0	75.9	72.8	87.8
Baseline	86.7	98.2	99.4	86.6	70.4	68.3	84.9
Ours	93.0	99.8	100.0	91.0	75.6	73.7	88.9

---

 | Show Table

DownLoad: CSV

The results for VisDA-2017 are shown in Table 4, where our model achieves an average accuracy of 83.0%. The method dynamically weighs the real-time estimation of transferability and discriminability during the iteration process, thus ensuring both aspects evolve towards an improved performance.

Table 4.  Results on VisDA-2017. (ResNet-50).

Methods	I$\to$P
ResNet-50[41]	74.8
WDGRL[42]	76.8
MCD[5]	77.3
DANN[2]	75.0
TCM[56]	75.8
CGDM[9]	82.3
Baseline	75.9
Ours	83.0

---

 | Show Table

DownLoad: CSV

In experiments conducted on all datasets, we perform a longitudinal comparison with the baseline. The experimental results demonstrate that our method fully takes the role of the domain information and class information during the model training process into account, thus effectively balancing the focus on the inter-domain distribution differences and the class discrimination capability. Our method achieves superior performance compared to the baseline, thus proving its effectiveness.

4.3.   Experimental analysis

Ablation Study: In this section, we analyze the impact of the Wasserstein Difference and the Dynamic Balancing Factor on the performance improvement. The results are shown in Table 5. We conduct ablation experiments on the Office-31 dataset across three tasks (A→D, D→W, W→A) by evaluating the effects of using or not using the Wasserstein Difference Loss in adversarial training and the Dynamic Balancing mechanism. In the second column of the table, the Wasserstein Difference Loss is replaced with MMD Loss for adversarial training. From the overall results in the table, it can be observed that removing both the Wasserstein Difference and the Dynamic Balancing mechanism negatively impacts the performance. Furthermore, it is evident that the use of the Wasserstein Difference Loss in the adversarial training of the discriminator has a more significant effect on the performance. Compared to the fixed balancing parameters, the Dynamic Balancing mechanism effectively maintains a better trade-off between the domain alignment and the classification, thus leading to consistent performance improvements.

Table 5.  Ablation experiment on office-31.

${L_w}$	MMD loss	Dynamic factor	fixed parameter	A$\to$D	D$\to$W	W$\to$A
×	√	×	5:5	86.6	98.2	68.3
√	×	×	5:5	90.1	99.6	73.1
×	√	√	-	88.6	99.2	72.8
√	×	√	-	91.0	99.8	73.7

---

 | Show Table

DownLoad: CSV

Analysis of Dynamic Weighting Parameters: To specifically analyze the effect of the dynamic weighting strategy, we conducted experiments using static parameter settings. The experimental results are shown in Table 6. From the data, it can be observed that the dynamic weighting strategy we employed achieves a better performance compared to the static hyperparameter settings, thus demonstrating its effectiveness in balancing the domain alignment and classification. As shown in Figure 3.6, we plot the classification error rate and the variation of the dynamic balancing parameter for the A→D task on the Office-31 dataset. It can be observed that as training progresses, the error rate of label prediction gradually converges. During the early stages of training, the domain information of the images is given a higher importance, thus enabling a rapid improvement in the domain alignment. In the middle stages of training, the dynamic parameter gradually increases, thus emphasizing the importance of the classification information, thereby further enhancing the classification process.

Table 6.  Accuracy (%) of different weight(A$\to$D).

weight	1: 9	3: 7	5: 5	7: 3	9: 1	$\alpha :(1-\alpha)$
Accuracy	90.3	89.4	90.1	90.6	88.6	91.0

---

 | Show Table

DownLoad: CSV

Confusion Matrix: The confusion matrix is shown in Figure 3. If the model is solely trained on the source data, then it suffers from severe class confusion, significantly reducing classification accuracy, as depicted in Figure 3(a). DANN overlooks the discriminability between features, thus solely focusing on domain alignment, which leads to a misclassification in certain categories, as shown in Figure 3(a). In our method, the main diagonal elements of the matrix exhibit the highest values, with significantly reduced off-diagonal elements, thus indicating the absence of a misclassification, as illustrated in Figure 3(c). This demonstrates the effectiveness of our approach.

Certainty is evaluated by calculating the ratio of correctly classified samples with high prediction confidence. Here, we consider the task A→R, where a prediction probability between 0.9 and 1 is deemed a high-confidence prediction. As shown in Figure 2(c), it is evident that a model trained solely on the source domain is almost incapable of generating a high ratio of confident predictions. DANN significantly increases the ratio of high-confidence predictions, and DMDA further improves it to 85.7%. However, all these methods have lower ratios compared to the method proposed in this chapter, thus demonstrating its effectiveness in enhancing the prediction certainty.

Figure 2.  Visualization of experimental data.

DownLoad: Full-Size Img PowerPoint

Figure 3.  Confusion matrices for different tasks.

DownLoad: Full-Size Img PowerPoint

t-SNE Visualization: The t-SNE visualization is shown in Figure 4. The DBDA proposed in this chapter results in more compact intra-class distributions and more dispersed inter-class distributions, thus indicating that the features learned by the DBDA possess a stronger discriminative power. The intra-class features are pulled together, while the inter-class features are pushed apart.

Figure 4.  Feature distribution t-SNE visualization learned on A→W tasks in Office-31. (The blue dots and red dots represent the characteristics of the source and target domains, respectively.).

DownLoad: Full-Size Img PowerPoint

5.   Conclusions

In our work, we proposed a simple yet effective adversarial paradigm that reused task-specific classifiers as discriminators. To implement this paradigm, we utilized a clearly guided difference, and accordingly established a discriminator-free adversarial UDA model. This model learns transferable and discriminative representations while ensuring the prediction certainty and diversity. Furthermore, based on a dynamic weighting algorithm, it efficiently adjusts the weights of domain alignment and class alignment during computation, making it suitable for various applications and achieving a dynamic balance. The test results on multiple samples indicated that the proposed algorithm performed well.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

This work was supported by the Open Fund of Key Laboratory of Philosophy and Social Science of Anhui Province on Adolescent Mental Health and Crisis Intelligence Intervention (SYS2024B05), Anhui Mine IOT and Security Monitoring Technology Key Laboratory(2109Y-09-04), Natural Science Research Project of Anhui Educational Committee(2022AH052144), Natural Science Research Project of Hefei Normal University(2022SKZD19).

Conflict of interest

The authors declare there is no conflicts of interest.