Machine Learning Workshop

In this 3 hours talk, Yang will talk about the basic properties of a linear model while emphasizing on its statistical properties. This talk will pave ways for the future talks to come, since linear model lays the foundation for various machine method and model selection criteria.

For some good resource for review, This document gives a concise review on regression concept from a more engineer point of view.

Time series model is rich both in its theory and history. In this talk we briefly talk about the concepts and challenges of dependent data, then quickly move on to time series - a very common type of dependent data.

Schwannden will introduce the classical models of time series (Liner model, VAR, and VARMA). We will walk through the basic mathematical principles of these various model, compare them with linear model, and talk about Granger causality.

In mixture model, Bayesian network, or other complex model with latent structure, our predictive distribution often can not be solved analytically. Method like EM (expectation maximizatrion) or variational inference provide us a way to approximate predictive distribution in a computationally acceptable manner.

Other method for approximation like sampling method will be given in later talks. But in practice, sampling methods can be compu tationally demanding, often limiting their use to small-scale problems. Also, it can be difficult to know whether a sampling scheme is generating independent samples from the required distribution.

Model selection is the task of selecting a statistical model from a set of candidate models, given data. In the simplest cases, a pre-existing set of data is considered. However, the task can also involve the design of experiments such that the data collected is well-suited to the problem of model selection. Given candidate models of similar predictive or explanatory power, the simplest model is most likely to be the best choice.

In 21st century, the advance of our computation power is not catching up with the increase in both amount and dimension of our data. This makes intelligent model selection a popular topic, and a practical challenge to our existing application.

For most probabilistic models of practical interest, exact inference is intractable, and so we have to resort to some form of approximation. In last talk, we discussed inference algorithms based on deterministic approximations, which include methods such as variational Bayes and expectation propagation. Here we consider approximate inference methods based on numerical sampling, also known as Monte Carlo techniques.

This this talk, we try to give the methematical background for sampling method, and give examples on MCMC, Gibb's sampling, rejection sampling, and Metropolis–Hastings.

In mixture of Gaussians, we discussed probabilistic models having discrete latent variables. We now explore models in which some, or all, of the latent variables are continuous. An important motivation for such models is that many data sets have the property that the data points all lie close to a manifold of much lower dimensionality than that of the original data space.

If you are a statistician, you are very attached to your p-values and saying variables have "significant effects." But in modren day model, we often do not have a analycally solvable statistics for us to do hypothesis testing. Bootstrap provides us a mean to do hypothesis testing even if we don't know the exact distribution of our statistics.