New Posts

content/blog/bandwidthlimitingapps.md

@@ -0,0 +1,36 @@
+---
+title: "Bandwidth Limiting Applications"
+date: 2020-10-22T21:51:27-04:00
+draft: false
+tags: []
+---
+
+Whether it's web scraping or testing low latency connections, we want to be able to bandwidth limit certain applications. I've written about [rate limiting network interfaces](/blog/limitbandwidth/) before. This post focuses on the application level instead.
+
+First if the application has an option to rate limit, then we should prefer that. Here are the ways you can rate limit in some common terminal applications. Let's say you set the bandwidth limit as `$max_kbps`
+
+```bash
+rsync --bwlimit $max_kbps ...
+```
+
+```bash
+curl --limit-rate $max_kbps ...
+```
+
+```bash
+wget --limit-rate $max_kbps ...
+```
+
+If the application you want to rate limit does not have a convenient flag and is dynamically linked, then we can use the application [`trickle`](https://github.com/mariusae/trickle).
+
+```bash
+trickle -s -d $max_kbps -u $max_kbps
+```
+
+| Flag | Description                                |
+| ---- | ------------------------------------------ |
+| `-s` | Standalone Mode                            |
+| `-d` | Download Bandwidth Consumption Rate (KB/s) |
+| `-u` | Upload Bandwidth Consumption Rate (KB/s)   |
+
+Trickle works by replacing the socket library that is dynamically linked. 

content/blog/quickcgroups.md

@@ -0,0 +1,51 @@
+---
+title: "Quickly Creating CGroups to Limit CPU/Memory of Applications"
+date: 2020-10-22T22:32:56-04:00
+draft: false
+tags: []
+---
+
+Running some data science algorithms can be CPU or memory intensive. To prevent the code from hogging system resources, we can use cgroups to set limits. This also is great for applications that you don't quite trust.
+
+## Defining the CGroup
+
+On Ubuntu, we will have to install `cgroup-tools`
+
+```bash
+sudo apt install cgroup-tools
+```
+
+Let's create a cgroup called `limitapp`
+
+```bash
+sudo cgcreate -a $USER -t $USER -g memory,cpu:limitapp
+```
+
+We can limit the RAM usage in this cgroup to 2GB. [According to the Arch Linux Wiki](https://wiki.archlinux.org/index.php/Cgroups#Ad-hoc_groups), this limit only applies to RAM not SWAP.
+
+```bash
+echo 2000000000 > /sys/fs/cgroup/memory/limitapp/memory.limit_in_bytes
+```
+
+All processes belong to a cgroup. The majority of them belong to what is considered the root cgroup. By default, all groups have 1024 CPU shares. The amount of shares that you define in the cgroup determines the approximate percentage of CPU time that the cgroup will get if the system is congested.
+
+Let's say that our cgroup can have 50% of the CPU time when the system is stressed:
+
+```bash
+echo 512 > /sys/fs/cgroup/cpu/limitapp/cpu.shares
+```
+
+## Assigning Applications to CGroups
+
+We can assign an existing application to a cgroup. For example, let us assign `firefox` to our cgroup.
+
+```bash
+cgclassify -g memory,cpu:limitapp $(pidof firefox)
+```
+
+We can start new processes in our cgroup. For example, let us start a python environment in our cgroup.
+
+```bash
+cgexec -g memory,cpu:limitapp python
+```
+