Supervised learning

Definition : Supervised learning

Artificial Neural Network

Supervised learning is a Machine Learning (ML) technique for approximating the input/output behavior of complex systems. The task of the supervised learner is to predict the output behavior of a system for any set of input values, after an initial trainig phase.


In supervised learning, there are two factors we can control for optimal generalization: sample points and models. A sample point corresponds to a query to the "oracle", and a model refers to, for example, the type and number of basis functions used for learning. Model Selection Model selection is one of the central research topics in supervised learning. It is important to determine the complexity of learning machines (models) appropriately for optimal generalization performance. If the model is too simple, it is not flexible enough to represent the learning target function and therefore the generalization performance can not be improved even if a large amount of training data is employed for learning. On the other hand, if the model is too complex, it is capable of representing the target function, but such a complex model is heavily affected by the noise in the training data. Therefore, in practical situations with a rather small number of training samples, complex models do not give better generalization performance. Active Learning The problem of selecting sample points is called active learning (also known as sample selection, sequential design, or optimal experimental design (OED)). If the training data set is designed appropriately, one can obtain better generalization capability even with a small amount of training data. [ref]

Research goals

We study and develop fully automated supervised learning techniques for fast and efficient multivariate regression, based on a limited & adaptively selected set of training data samples. The model type (e.g., ANN, SVM, rational model, ...), model complexity (e.g., number of neurons and hidden layers, kernel function, order, ...), and model parameters (e.g., using back-propagation, least-squares approximation, ...) are fully automatically selected.

This research is based on, or linked with:
  • Artificial Intelligence (AI)
    [Computational Intelligence, Machine Learning (ML), Supervised Learning, Active Learning, Artificial Neural Networks (ANN), Genetic Algorithms (GA), Evolutionary Computing (EC), Adaptive/Sequential sampling, Reflective Exploration (RE), Knowledge discovery]

  • Experimental Design / Computer-aided Design
    [Design Of Experiments (DOE), Response Surface Modeling (RSM), Design and Analysis of Computer Experiments (DACE), Kriging methods, Metamodeling , Data-driven information processing]

  • Numerical techniques
    [Data-driven modelling, Multivariate interpolation and approximation, Rational functions, Radial Basis Functions (RBF), Scattered data interpolation, Orthonormal bases for parameter estimation, Model Order Reduction (MOR), Model Based Parameter Estimation (MBPE), Optimization]