website/static/~brozek/index.html?research%2FClusterAnalysis%2Fnotes%2Flec2-2.html

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8" />
  <meta name="author" content="Brandon Rozek">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <meta name="robots" content="noindex" />
    <title>Brandon Rozek</title>
  <link rel="stylesheet" href="themes/bitsandpieces/styles/main.css" type="text/css" />
  <link rel="stylesheet" href="themes/bitsandpieces/styles/highlightjs-github.css" type="text/css" />
</head>
<body>

<aside class="main-nav">
<nav>
  <ul>
          <li class="menuitem ">
        <a href="index.html%3Findex.html" data-shortcut="">
          Home
                  </a>
      </li>
          <li class="menuitem ">
        <a href="index.html%3Fcourses.html" data-shortcut="">
          Courses
                  </a>
      </li>
          <li class="menuitem ">
        <a href="index.html%3Flabaide.html" data-shortcut="">
          Lab Aide
                  </a>
      </li>
          <li class="menuitem ">
        <a href="index.html%3Fpresentations.html" data-shortcut="">
          Presentations
                  </a>
      </li>
          <li class="menuitem ">
        <a href="index.html%3Fresearch.html" data-shortcut="">
          Research
                  </a>
      </li>
          <li class="menuitem ">
        <a href="index.html%3Ftranscript.html" data-shortcut="">
          Transcript
                  </a>
      </li>
      </ul>
</nav>
</aside>
<main class="main-content">
  <article class="article">
    <h1>Principal Component Analysis Pt. 1</h1>
<h2>What is PCA?</h2>
<p>Principal component analysis is a statistical procedure that performs an orthogonal transformation to convert a set of variables into a set of linearly uncorrelated variables called principle components.</p>
<p>Number of distinct principle components equals $min(# Variables, # Observations - 1)$</p>
<p>The transformation is defined in such a way that the first principle component has the largest possible variance explained in the data.</p>
<p>Each succeeding component has the highest possible variance under the constraint of having to be orthogonal to the preceding components.</p>
<p>PCA is sensitive to the relative scaling of the original variables.</p>
<h3>Results of a PCA</h3>
<p>Results are discussed in terms of <em>component scores</em> which is the transformed variables and <em>loadings</em> which is the weight by which each original variable should be multiplied to get the component score.</p>
<h2>Assumptions of PCA</h2>
<ol>
<li>Linearity</li>
<li>Large variances are important and small variances denote noise</li>
<li>Principal components are orthogonal</li>
</ol>
<h2>Why perform PCA?</h2>
<ul>
<li>Distance measures perform poorly in high-dimensional space (<a href="https://stats.stackexchange.com/questions/256172/why-always-doing-dimensionality-reduction-before-clustering">https://stats.stackexchange.com/questions/256172/why-always-doing-dimensionality-reduction-before-clustering</a>)</li>
<li>Helps eliminates noise from the dataset (<a href="https://www.quora.com/Does-it-make-sense-to-perform-principal-components-analysis-before-clustering-if-the-original-data-has-too-many-dimensions-Is-it-theoretically-unsound-to-try-to-cluster-data-with-no-correlation">https://www.quora.com/Does-it-make-sense-to-perform-principal-components-analysis-before-clustering-if-the-original-data-has-too-many-dimensions-Is-it-theoretically-unsound-to-try-to-cluster-data-with-no-correlation</a>)</li>
<li>One initial cost to help reduce further computations</li>
</ul>
<h2>Computing PCA</h2>
<ol>
<li>Subtract off the mean of each measurement type</li>
<li>Compute the covariance matrix</li>
<li>Take the eigenvalues/vectors of the covariance matrix</li>
</ol>
<h2>R Code</h2>
<pre><code class="language-R">pcal = function(data) {
  centered_data = scale(data)
  covariance = cov(centered_data)
  eigen_stuff = eigen(covariance)
  sorted_indices = sort(eigen_stuff$values, 
                        index.return = T, 
                        decreasing = T)$ix
  loadings = eigen_stuff$values[sorted_indices]
  components = eigen_stuff$vectors[sorted_indices,]
  combined_list = list(loadings, components)
  names(combined_list) = c("Loadings", "Components")
  return(combined_list)
}</code></pre>
  </article>
</main>

<script src="themes/bitsandpieces/scripts/highlight.js"></script>
<script src="themes/bitsandpieces/scripts/mousetrap.min.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    tex2jax: {
      inlineMath: [ ['$','$'], ["\\(","\\)"] ],
      processEscapes: true
    }
  });
</script>

<script type="text/javascript"
    src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<script>
  hljs.initHighlightingOnLoad();
  
  document.querySelectorAll('.menuitem a').forEach(function(el) {
    if (el.getAttribute('data-shortcut').length > 0) {
      Mousetrap.bind(el.getAttribute('data-shortcut'), function() {
        location.assign(el.getAttribute('href'));
      });       
    }
  });
</script>

</body>
</html>
Added old tildasite as an archive 2020-01-16 04:07:02 +00:00			`<!DOCTYPE html>`
			`<html>`
			`<head>`
			`<meta charset="utf-8" />`
Fixed meta tags 2022-02-15 06:14:58 +00:00			`<meta name="author" content="Brandon Rozek">`
Added old tildasite as an archive 2020-01-16 04:07:02 +00:00			`<meta name="viewport" content="width=device-width, initial-scale=1.0">`
			`<meta name="robots" content="noindex" />`
			`<title>Brandon Rozek</title>`
			`<link rel="stylesheet" href="themes/bitsandpieces/styles/main.css" type="text/css" />`
			`<link rel="stylesheet" href="themes/bitsandpieces/styles/highlightjs-github.css" type="text/css" />`
			`</head>`
			`<body>`

			`<aside class="main-nav">`
			`<nav>`
			`<ul>`
			`<li class="menuitem ">`
			`<a href="index.html%3Findex.html" data-shortcut="">`
			`Home`
			`</a>`
			`</li>`
			`<li class="menuitem ">`
			`<a href="index.html%3Fcourses.html" data-shortcut="">`
			`Courses`
			`</a>`
			`</li>`
			`<li class="menuitem ">`
			`<a href="index.html%3Flabaide.html" data-shortcut="">`
			`Lab Aide`
			`</a>`
			`</li>`
			`<li class="menuitem ">`
			`<a href="index.html%3Fpresentations.html" data-shortcut="">`
			`Presentations`
			`</a>`
			`</li>`
			`<li class="menuitem ">`
			`<a href="index.html%3Fresearch.html" data-shortcut="">`
			`Research`
			`</a>`
			`</li>`
			`<li class="menuitem ">`
			`<a href="index.html%3Ftranscript.html" data-shortcut="">`
			`Transcript`
			`</a>`
			`</li>`
			`</ul>`
			`</nav>`
			`</aside>`
			`<main class="main-content">`
			`<article class="article">`
			`<h1>Principal Component Analysis Pt. 1</h1>`
			`<h2>What is PCA?</h2>`
			`<p>Principal component analysis is a statistical procedure that performs an orthogonal transformation to convert a set of variables into a set of linearly uncorrelated variables called principle components.</p>`
			`<p>Number of distinct principle components equals $min(# Variables, # Observations - 1)$</p>`
			`<p>The transformation is defined in such a way that the first principle component has the largest possible variance explained in the data.</p>`
			`<p>Each succeeding component has the highest possible variance under the constraint of having to be orthogonal to the preceding components.</p>`
			`<p>PCA is sensitive to the relative scaling of the original variables.</p>`
			`<h3>Results of a PCA</h3>`
			`<p>Results are discussed in terms of <em>component scores</em> which is the transformed variables and <em>loadings</em> which is the weight by which each original variable should be multiplied to get the component score.</p>`
			`<h2>Assumptions of PCA</h2>`
			`<ol>`
			`<li>Linearity</li>`
			`<li>Large variances are important and small variances denote noise</li>`
			`<li>Principal components are orthogonal</li>`
			`</ol>`
			`<h2>Why perform PCA?</h2>`
			`<ul>`
			`<li>Distance measures perform poorly in high-dimensional space (<a href="https://stats.stackexchange.com/questions/256172/why-always-doing-dimensionality-reduction-before-clustering">https://stats.stackexchange.com/questions/256172/why-always-doing-dimensionality-reduction-before-clustering</a>)</li>`
			`<li>Helps eliminates noise from the dataset (<a href="https://www.quora.com/Does-it-make-sense-to-perform-principal-components-analysis-before-clustering-if-the-original-data-has-too-many-dimensions-Is-it-theoretically-unsound-to-try-to-cluster-data-with-no-correlation">https://www.quora.com/Does-it-make-sense-to-perform-principal-components-analysis-before-clustering-if-the-original-data-has-too-many-dimensions-Is-it-theoretically-unsound-to-try-to-cluster-data-with-no-correlation</a>)</li>`
			`<li>One initial cost to help reduce further computations</li>`
			`</ul>`
			`<h2>Computing PCA</h2>`
			`<ol>`
			`<li>Subtract off the mean of each measurement type</li>`
			`<li>Compute the covariance matrix</li>`
			`<li>Take the eigenvalues/vectors of the covariance matrix</li>`
			`</ol>`
			`<h2>R Code</h2>`
			`<pre><code class="language-R">pcal = function(data) {`
			`centered_data = scale(data)`
			`covariance = cov(centered_data)`
			`eigen_stuff = eigen(covariance)`
			`sorted_indices = sort(eigen_stuff$values,`
			`index.return = T,`
			`decreasing = T)$ix`
			`loadings = eigen_stuff$values[sorted_indices]`
			`components = eigen_stuff$vectors[sorted_indices,]`
			`combined_list = list(loadings, components)`
			`names(combined_list) = c("Loadings", "Components")`
			`return(combined_list)`
			`}</code></pre>`
			`</article>`
			`</main>`

			`<script src="themes/bitsandpieces/scripts/highlight.js"></script>`
			`<script src="themes/bitsandpieces/scripts/mousetrap.min.js"></script>`
			`<script type="text/x-mathjax-config">`
			`MathJax.Hub.Config({`
			`tex2jax: {`
			`inlineMath: [ ['$','$'], ["\\(","\\)"] ],`
			`processEscapes: true`
			`}`
			`});`
			`</script>`

			`<script type="text/javascript"`
			`src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML">`
			`</script>`
			`<script>`
			`hljs.initHighlightingOnLoad();`

			`document.querySelectorAll('.menuitem a').forEach(function(el) {`
			`if (el.getAttribute('data-shortcut').length > 0) {`
			`Mousetrap.bind(el.getAttribute('data-shortcut'), function() {`
			`location.assign(el.getAttribute('href'));`
			`});`
			`}`
			`});`
			`</script>`

			`</body>`
			`</html>`