New Approaches in Learning Complex-Valued Neural Networks
This study explores innovative methods in training complex-valued neural networks, including a model of complex-valued neurons, network architecture, error analysis, Adam optimizer, gradient calculation, and activation function selection. Simulation results compare real-valued and complex-valued networks in solving the hard XOR problem, demonstrating improved performance with new activation functions and optimizers like CReLU and Adam. The conclusions highlight the efficacy of feed-forward complex-valued neural networks for nonlinear mappings and suggest further exploration to optimize learning frameworks.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
NEW APPROACHES IN THE LEARNING OF COMPLEX-VALUED NEURAL NETWORKS V L A D Y S L A V K O T S O V S K Y , A N A T O L I Y B A T Y U K , M A K S Y M Y U R C H E N K O
SIMULATION We applied the complex backpropagation algorithm for solving the hard XOR problem in the case of 50features. We compared the performance of the real-valued and complex-valued four-layer networks consisting of neurons with sigmoidal and ReLU-like activation functions. We used 50-20- 10-1 topology in the real case and 25-20-10-1 topology in the complex case. Each network was trained in on-line mode on 100 different learning samples. The number of epochs was equal to 1000. We use q = 1 for complex rational sigmoid. 9
SIMULATION The parameter 1 and 2 were initialized to 0.9 and 0.999, respectively. The following presents average performance mea- sures in the case of learning using stochastic gradient descent (SGD) and Adam optimizers, respectively. Average accuracy (in %) Activation function Training set SGD Test set SGD Training set Adam Test set Adam Real sigmoid 68.15 53.24 74.03 62.12 Splitted complex sigmoid 71.92 62.25 82.88 67.03 Complex rational sigmoid 73.01 62.19 85.35 67.76 CReLU 74.96 67.05 92.43 81.51 10
CONCLUSIONS Feed-forward complex-valued neural networks give a simple and powerful architecture approximation of nonlinear mappings. The new activation functions and the complex modifications of effective modern optimizers improve the quality of learning. Using the results of simulation, we can conclude that the combination of CReLU activation and Adam optimizer is preferable under conditions of our experiment. The new theoretical study and experiments are needed for tuning better parameters in the learning framework and establishing the behavior for such networks. providing an adequate 11
