<!DOCTYPE html> <html> <head> <meta charset="utf-8" /> <meta name="author" content="Brandon Rozek"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="robots" content="noindex" /> <title>Brandon Rozek</title> <link rel="stylesheet" href="themes/bitsandpieces/styles/main.css" type="text/css" /> <link rel="stylesheet" href="themes/bitsandpieces/styles/highlightjs-github.css" type="text/css" /> </head> <body> <aside class="main-nav"> <nav> <ul> <li class="menuitem "> <a href="index.html%3Findex.html" data-shortcut=""> Home </a> </li> <li class="menuitem "> <a href="index.html%3Fcourses.html" data-shortcut=""> Courses </a> </li> <li class="menuitem "> <a href="index.html%3Flabaide.html" data-shortcut=""> Lab Aide </a> </li> <li class="menuitem "> <a href="index.html%3Fpresentations.html" data-shortcut=""> Presentations </a> </li> <li class="menuitem "> <a href="index.html%3Fresearch.html" data-shortcut=""> Research </a> </li> <li class="menuitem "> <a href="index.html%3Ftranscript.html" data-shortcut=""> Transcript </a> </li> </ul> </nav> </aside> <main class="main-content"> <article class="article"> <h1>Principal Component Analysis Pt. 1</h1> <h2>What is PCA?</h2> <p>Principal component analysis is a statistical procedure that performs an orthogonal transformation to convert a set of variables into a set of linearly uncorrelated variables called principle components.</p> <p>Number of distinct principle components equals $min(# Variables, # Observations - 1)$</p> <p>The transformation is defined in such a way that the first principle component has the largest possible variance explained in the data.</p> <p>Each succeeding component has the highest possible variance under the constraint of having to be orthogonal to the preceding components.</p> <p>PCA is sensitive to the relative scaling of the original variables.</p> <h3>Results of a PCA</h3> <p>Results are discussed in terms of <em>component scores</em> which is the transformed variables and <em>loadings</em> which is the weight by which each original variable should be multiplied to get the component score.</p> <h2>Assumptions of PCA</h2> <ol> <li>Linearity</li> <li>Large variances are important and small variances denote noise</li> <li>Principal components are orthogonal</li> </ol> <h2>Why perform PCA?</h2> <ul> <li>Distance measures perform poorly in high-dimensional space (<a href="https://stats.stackexchange.com/questions/256172/why-always-doing-dimensionality-reduction-before-clustering">https://stats.stackexchange.com/questions/256172/why-always-doing-dimensionality-reduction-before-clustering</a>)</li> <li>Helps eliminates noise from the dataset (<a href="https://www.quora.com/Does-it-make-sense-to-perform-principal-components-analysis-before-clustering-if-the-original-data-has-too-many-dimensions-Is-it-theoretically-unsound-to-try-to-cluster-data-with-no-correlation">https://www.quora.com/Does-it-make-sense-to-perform-principal-components-analysis-before-clustering-if-the-original-data-has-too-many-dimensions-Is-it-theoretically-unsound-to-try-to-cluster-data-with-no-correlation</a>)</li> <li>One initial cost to help reduce further computations</li> </ul> <h2>Computing PCA</h2> <ol> <li>Subtract off the mean of each measurement type</li> <li>Compute the covariance matrix</li> <li>Take the eigenvalues/vectors of the covariance matrix</li> </ol> <h2>R Code</h2> <pre><code class="language-R">pcal = function(data) { centered_data = scale(data) covariance = cov(centered_data) eigen_stuff = eigen(covariance) sorted_indices = sort(eigen_stuff$values, index.return = T, decreasing = T)$ix loadings = eigen_stuff$values[sorted_indices] components = eigen_stuff$vectors[sorted_indices,] combined_list = list(loadings, components) names(combined_list) = c("Loadings", "Components") return(combined_list) }</code></pre> </article> </main> <script src="themes/bitsandpieces/scripts/highlight.js"></script> <script src="themes/bitsandpieces/scripts/mousetrap.min.js"></script> <script type="text/x-mathjax-config"> MathJax.Hub.Config({ tex2jax: { inlineMath: [ ['$','$'], ["\\(","\\)"] ], processEscapes: true } }); </script> <script type="text/javascript" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"> </script> <script> hljs.initHighlightingOnLoad(); document.querySelectorAll('.menuitem a').forEach(function(el) { if (el.getAttribute('data-shortcut').length > 0) { Mousetrap.bind(el.getAttribute('data-shortcut'), function() { location.assign(el.getAttribute('href')); }); } }); </script> </body> </html>