NeuroSolutions

by NeuroDimension

NeuroSolutions is a highly graphical neural network development tool for Windows 95/98/NT/2000/XP. This leading edge software combines a modular, icon-based network design interface with an implementation of advanced learning procedures and genetic optimization. The result is a virtually unconstrained environment for designing neural networks for research and for solving real-world problems.

The Latest in Neural Network Technologies

Our strong ties with the world-renowned Computational Neural Engineering Lab at the University of Florida enable us to keep up with the state-of-the-art in neural network technology and incorporate it into the software. Here are some of the most recent additions to NeuroSolutions:

• Neuro-Fuzzy - The coactive neuro-fuzzy inference system (CANFIS) model integrates fuzzy inputs with a neural network to quickly solve poorly defined problems. Fuzzy inference systems are also valuable as they combine the explanatory nature of rules (membership functions) with the power of neural networks.
   
• Support Vector Machine - The Support Vector Machine (SVM) model maps inputs to a high-dimensional feature space, and then optimally separates data into their respective classes by isolating those inputs that fall close to the data boundaries. They are especially effective in separating sets of data that share complex boundaries.
   
• Conjugate Gradient - Conjugate gradient learning is a second-order training method that provides an excellent trade-off between complexity and performance. Typically it trains faster and better (lower MSE) than standard backpropagation. In addition, it is completely parameterless -- no learning rates or momentum terms to adjust.
   
• Teacher Forcing / Iterative Prediction - There are some time-series prediction problems that are best modeled using a method called teacher forcing. This specialized training algorithm feeds the predicted output back into the input in order to improve the accuracy of multi-step prediction. The predicted output of networks trained with teacher forcing is then obtained using iterative prediction.

Temporal Neural Networks

NeuroSolutions is one of the few neural network development tools to fully support backpropagation through time (BPTT). Instead of mapping a static input to a static output, BPTT maps a series of inputs to a series of outputs. This provides the ability to solve temporal problems by extracting how data changes over time. Examples of temporal problems are digital signal processing, speech recognition, and time-series prediction.

User-defined Neural Topologies

NeuroSolutions is based on the concept that neural networks can be broken down into a fundamental set of neural components. Individually these components are relatively simplistic, but several components connected together can result in networks capable of solving very complex problems. The network construction wizards will connect these components for you based on your specifications. However, once the network is built you can arbitrarily change interconnections and/or add in new components. In other words, a virtually infinite number of neural models are possible!

User-defined Neural Components

The Developers and Developers Lite levels allow you to integrate your own algorithms into NeuroSolutions through dynamic link libraries (DLLs). Every NeuroSolutions component implements a function conforming to a simple protocol in C. To add a new component you simply modify the template function for the base component and compile the code into a DLL -- all directly from NeuroSolutions! Note: this feature requires that Microsoft Visual C++ (version 5.0 or higher) be installed on the same machine.

C++ Code Generation

An application developer can integrate a NeuroSolutions neural network into their application by generating a DLL with the Custom Solution Wizard or by generating the C++ source code for the network using the Professional or Developers level of NeuroSolutions. The source code generation facility of NeuroSolutions is as robust as its object-oriented design environment. No matter how simple or complex of a network you create within the graphical user interface, NeuroSolutions will generate the equivalent neural network in ANSI C++ source code -- even those networks that contain your own algorithms implemented with DLLs! The generated network can be trained beforehand within the graphical design environment of NeuroSolutions or from within your C++ application.

The generated code links with a pre-compiled object library, which contains all of the neural component algorithms provided by NeuroSolutions. NeuroSolutions ships with the libraries for the Microsoft Visual C++ (5.0 and above) and Borland Builder (3.0 and above) compilers. However, some developers may wish to use a different Windows compiler, or a compiler on a different operating system entirely, such as UNIX. For this reason, NeuroDimension makes the source code for the entire object library available as a separate product called the Source Code License. This license also gives you the ultimate level of flexibility in that you can modify the component library code to meet your unique needs.

The license for the object library, as well as the Source Code License, entitles the licensee to distribute recall networks royalty-free. Learning networks are restricted to one machine per license, although royalty agreements for distribution can be formulated on a case-by-case basis. Note that learning DLLs generated with the Custom Solution Wizard are not subject to royalties.

Extensive Probing Capabilities

Neural networks are often criticized as being a "black box" technology. With NeuroSolutions' extensive and versatile set of probing tools, this is no longer the case. Probes provide you with real-time access to all internal network variables, such as:

• Inputs/Outputs
• Weights
• Errors
• Hidden States
• Gradients
• Sensitivities

Probing is an important step in the neural network design process, therefore we have made it an integral part of NeuroSolutions. As with the neural components, the probe components are inherently modular; the way you view the data is independent of what the data represents. All network data are reported through a common protocol, and all NeuroSolutions probes understand this protocol. This provides you with access to all internal variables, along with a variety of ways to visualize them.

Genetic Optimization

The Users level of NeuroSolutions and above include Genetic Optimization. Genetic Optimization allows you to optimize virtually any parameter in a neural network to produce the lowest error. For example, the number of hidden units, the learning rates, and the input selection can all be optimized to improve the network performance. Individual weights used in the neural network can even be updated through Genetic Optimization as an alternative to traditional training methods.

Sensitivity Analysis

After training a neural network, you may want to know the effect that each of the network inputs is having on the network output. The sensitivity analysis feature of NeuroSolutions can be used to perform this function. Sensitivity analysis is a method for extracting the cause and effect relationship between the inputs and outputs of the network. The basic idea is that each input channel to the network is offset slightly and the corresponding change in the output(s) is reported. The input channels that produce low sensitivity values can be considered insignificant and can most often be removed from the network. This will reduce the size of the network, which in turn reduces the complexity and the training time. Furthermore, this will likely also improve the network performance for the out-of-sample testing data.

Exemplar Weighting

Classification problems often do not have an equal number of training exemplars (samples) for each class. For example, you may have a neural network application that detects the occurrence of cancer from clinical test data. The training data for this problem may contain 99 exemplars classified as non-cancerous for every one exemplar classified as cancerous. A standard neural network would most often train itself to classify all exemplars as non-cancerous so that it would be 99% correct. Since the goal is to detect the existence of cancer, this is a problem.

One way to overcome this problem would be to throw away most of the training exemplars so that there would be an equal number for each class. This would drastically reduce the amount of training data, and likely result in a network with poor generalization.

NeuroSolutions provides a better solution using a method called exemplar weighting. For the example above, each of the cancerous training exemplars would have 99 times more weight during the backpropagation procedure than the non cancerous exemplars. This balancing of the training data will most likely result in a system that does a much better job of detecting the cancerous cases.

Macros

NeuroSolutions has a comprehensive macro language, which allows the user to record a sequence of operations and store them as a program. Any action that can be performed using the mouse and keyboard can be duplicated with a macro statement. This powerful feature gives the user unprecedented flexibility in constructing, editing, and running neural networks. When running the demos of the evaluation version of NeuroSolutions, keep in mind that they were constructed entirely with macros.

OLE Automation

NeuroSolutions is a fully compliant OLE Automation Server. This means that NeuroSolutions can receive control messages from OLE Automation Controllers, such as Visual C++, Visual Basic, Microsoft Excel, Microsoft Access, and Delphi.

Writing a fully-functioning VB program is as simple as recording a NeuroSolutions macro, clicking the "Convert to VB" button, and pasting the converted VB code into the desired VB application. A VB application might be written to set a network?s parameters, run the network, then retrieve the network?s output. When running the demos of the evaluation version of NeuroSolutions, be sure to run the OLE Automation demo which shows a sample VB application that communicates with NeuroSolutions via OLE.

Network Construction Wizards

NeuroSolutions has two separate wizards that you can use to automatically build a neural network to your design specifications:

• NeuralExpert
  The NeuralExpert centers the design specifications around the type of problem you wish the neural network to solve (Classification, Prediction, Function Approximation or Clustering). Given this problem type and the size of your data set, the NeuralExpert intelligently selects the neural network size and architecture that will most likely produce a good solution. There is an optional beginner level that hides some of the more advanced operations such as cross validation and genetic optimization.
  
  
  

• NeuralBuilder
  The NeuralBuilder centers the design specifications around the specific neural network architecture you wish to have built. Some of the most common architectures include:
   

  Multilayer Perceptron (MLP) Generalized Feedforward Modular Jordan/Elman Principal Component Analysis (PCA) Radial Basis Function (RBF) General Regression Neural Network (GRNN)

  Probabilistic Neural Network (PNN) Self-Organizing Map (SOM) Time-Lag Recurrent Network (TLRN) Recurrent Network CANFIS Network (Fuzzy Logic) Support Vector Machine (SVM)

  Once you select the architecture you can customize parameters such as the number of hidden layers, the number of processing elements and the learning algorithm. If you don't know what a parameter should be set to, you can specify that a genetic algorithm be used to optimize the setting for you.
  

System Requirements:

• Windows 95/98/ME/2000/NT/XP
• 16MB RAM
• 40MB free disk space