website/static/~brozek/index.html?research%2FClusterAnalysis%2Fnotes%2Flec2-2.html

124 lines
5 KiB
HTML
Raw Permalink Normal View History

2020-01-15 23:07:02 -05:00
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
2022-02-15 01:14:58 -05:00
<meta name="author" content="Brandon Rozek">
2020-01-15 23:07:02 -05:00
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="robots" content="noindex" />
<title>Brandon Rozek</title>
<link rel="stylesheet" href="themes/bitsandpieces/styles/main.css" type="text/css" />
<link rel="stylesheet" href="themes/bitsandpieces/styles/highlightjs-github.css" type="text/css" />
</head>
<body>
<aside class="main-nav">
<nav>
<ul>
<li class="menuitem ">
<a href="index.html%3Findex.html" data-shortcut="">
Home
</a>
</li>
<li class="menuitem ">
<a href="index.html%3Fcourses.html" data-shortcut="">
Courses
</a>
</li>
<li class="menuitem ">
<a href="index.html%3Flabaide.html" data-shortcut="">
Lab Aide
</a>
</li>
<li class="menuitem ">
<a href="index.html%3Fpresentations.html" data-shortcut="">
Presentations
</a>
</li>
<li class="menuitem ">
<a href="index.html%3Fresearch.html" data-shortcut="">
Research
</a>
</li>
<li class="menuitem ">
<a href="index.html%3Ftranscript.html" data-shortcut="">
Transcript
</a>
</li>
</ul>
</nav>
</aside>
<main class="main-content">
<article class="article">
<h1>Principal Component Analysis Pt. 1</h1>
<h2>What is PCA?</h2>
<p>Principal component analysis is a statistical procedure that performs an orthogonal transformation to convert a set of variables into a set of linearly uncorrelated variables called principle components.</p>
<p>Number of distinct principle components equals $min(# Variables, # Observations - 1)$</p>
<p>The transformation is defined in such a way that the first principle component has the largest possible variance explained in the data.</p>
<p>Each succeeding component has the highest possible variance under the constraint of having to be orthogonal to the preceding components.</p>
<p>PCA is sensitive to the relative scaling of the original variables.</p>
<h3>Results of a PCA</h3>
<p>Results are discussed in terms of <em>component scores</em> which is the transformed variables and <em>loadings</em> which is the weight by which each original variable should be multiplied to get the component score.</p>
<h2>Assumptions of PCA</h2>
<ol>
<li>Linearity</li>
<li>Large variances are important and small variances denote noise</li>
<li>Principal components are orthogonal</li>
</ol>
<h2>Why perform PCA?</h2>
<ul>
<li>Distance measures perform poorly in high-dimensional space (<a href="https://stats.stackexchange.com/questions/256172/why-always-doing-dimensionality-reduction-before-clustering">https://stats.stackexchange.com/questions/256172/why-always-doing-dimensionality-reduction-before-clustering</a>)</li>
<li>Helps eliminates noise from the dataset (<a href="https://www.quora.com/Does-it-make-sense-to-perform-principal-components-analysis-before-clustering-if-the-original-data-has-too-many-dimensions-Is-it-theoretically-unsound-to-try-to-cluster-data-with-no-correlation">https://www.quora.com/Does-it-make-sense-to-perform-principal-components-analysis-before-clustering-if-the-original-data-has-too-many-dimensions-Is-it-theoretically-unsound-to-try-to-cluster-data-with-no-correlation</a>)</li>
<li>One initial cost to help reduce further computations</li>
</ul>
<h2>Computing PCA</h2>
<ol>
<li>Subtract off the mean of each measurement type</li>
<li>Compute the covariance matrix</li>
<li>Take the eigenvalues/vectors of the covariance matrix</li>
</ol>
<h2>R Code</h2>
<pre><code class="language-R">pcal = function(data) {
centered_data = scale(data)
covariance = cov(centered_data)
eigen_stuff = eigen(covariance)
sorted_indices = sort(eigen_stuff$values,
index.return = T,
decreasing = T)$ix
loadings = eigen_stuff$values[sorted_indices]
components = eigen_stuff$vectors[sorted_indices,]
combined_list = list(loadings, components)
names(combined_list) = c("Loadings", "Components")
return(combined_list)
}</code></pre>
</article>
</main>
<script src="themes/bitsandpieces/scripts/highlight.js"></script>
<script src="themes/bitsandpieces/scripts/mousetrap.min.js"></script>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
tex2jax: {
inlineMath: [ ['$','$'], ["\\(","\\)"] ],
processEscapes: true
}
});
</script>
<script type="text/javascript"
src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<script>
hljs.initHighlightingOnLoad();
document.querySelectorAll('.menuitem a').forEach(function(el) {
if (el.getAttribute('data-shortcut').length > 0) {
Mousetrap.bind(el.getAttribute('data-shortcut'), function() {
location.assign(el.getAttribute('href'));
});
}
});
</script>
</body>
</html>