From c01a9d6910c865342b08d7f7d2d32d45769d0dc5 Mon Sep 17 00:00:00 2001 From: Brandon Rozek Date: Fri, 10 Apr 2020 12:07:50 -0400 Subject: [PATCH 1/6] New Post and edited offlinepip --- content/blog/offlinepip.md | 2 ++ content/blog/pipconf.md | 42 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+) create mode 100644 content/blog/pipconf.md diff --git a/content/blog/offlinepip.md b/content/blog/offlinepip.md index ff6059f..abd652a 100644 --- a/content/blog/offlinepip.md +++ b/content/blog/offlinepip.md @@ -79,3 +79,5 @@ Or they can just install the packages they want pip install --no-index -f /path/to/wheels/wheels package_name ``` +If you don't want to add flags to every command, check out my post on using [configuration files with pip](https://brandonrozek.com/blog/pipconf/). + diff --git a/content/blog/pipconf.md b/content/blog/pipconf.md new file mode 100644 index 0000000..fa64914 --- /dev/null +++ b/content/blog/pipconf.md @@ -0,0 +1,42 @@ +--- +title: "Pip Config" +date: 2020-04-10T11:56:19-04:00 +draft: false +tags: ["python"] +--- + +If you find yourself added flags to every pip command, consider adding those flag's to a pip configuration file. + +In order of importance, the configuration files will be located + +- Inside the virtualenv `/path/to/virtualenv/pip.conf` +- In the user folder `~/.config/pip/pip.conf` +- Site-wide `/etc/pip.conf` + +It is structured as an INI file where the blocks are the commands (`global` indicates all commands) + +For an example, we can set the timeout for all commands to 60 seconds, but the timeout for the freeze command to only 10 seconds. + +```ini +[global] +timeout = 60 + +[freeze] +timeout = 10 +``` + +Boolean flags are set by assigning a value of `true` or `yes` to them + +```ini +[install] +ignore-installed = true +``` + +For operating in an offline environment, + +```ini +[global] +no-index = true +find-links = /path/to/wheels +``` + From 33195f5d72777bfd78c38898b7c98ccccc91fede Mon Sep 17 00:00:00 2001 From: Brandon Rozek Date: Tue, 14 Apr 2020 08:42:49 -0400 Subject: [PATCH 2/6] New Post --- content/blog/pysubscribepattern.md | 127 +++++++++++++++++++++++++++++ 1 file changed, 127 insertions(+) create mode 100644 content/blog/pysubscribepattern.md diff --git a/content/blog/pysubscribepattern.md b/content/blog/pysubscribepattern.md new file mode 100644 index 0000000..9a0ed6b --- /dev/null +++ b/content/blog/pysubscribepattern.md @@ -0,0 +1,127 @@ +--- +title: "Python Patterns: Subscribe" +date: 2020-04-14T07:53:46-04:00 +draft: false +tags: [] +--- + +It is common for larger applications to have modules that publishes and subscribes to events. This post will outline a couple ways to achieve this using [decorators](https://brandonrozek.com/blog/pydecorators/). + +## Single Event + +First let us concern ourselves with a single event since that's the easiest. Here we will create an application class that stores callbacks of functions through the subscribe decorator. Calling `emit` will send a message to all the functions stored in `self.callbacks`. + +```python +class Application: + def __init__(self): + self.callbacks = [] + def subscribe(self, func): + self.callbacks.append(func) + return func + def emit(self, message): + for callback in self.callbacks: + callback(message) +``` + +Here is an example of its usage: + +```python +app = Application() + +@app.subscribe +def test1(message): + print("Function 1:", message) + +@app.subscribe +def test2(message): + print("Function 2:", message) + +app.emit('Hello World') +``` + +``` +Function 1: Hello World +Function 2: Hello World +``` + +## Multiple Events + +Let's say you want the application to handle different types of events. Now `self.callbacks` is a dictionary of lists, where the key is the event and the list is the same as the last section. There's an additional layered function on top of `subscribe` this time in order to handle passing an argument into the decorator. + +```python +from collections import defaultdict + +class Application: + def __init__(self): + self.callbacks = defaultdict(list) + def on(self, event): + def subscribe(func): + self.callbacks[event].append(func) + return func + return subscribe + def emit(self, event, message): + for callback in self.callbacks[event]: + callback(message) +``` + +To show its usage lets first create an instance of `Application` + +```python +app = Application() +``` + +Now let's subscribe a couple functions to `event1` + +```python +@app.on('event1') +def test1(message): + print("Function 1:", message) + +@app.on('event1') +def test3(message): + print("Function 3:", message) +``` + +Now to subscribe a couple events to `event2` + +```python +# Subscribed to event 2 +@app.on('event2') +def test2(message): + print("Function 2:", message) + +@app.on('event2') +def test4(message): + print("Function 4:", message) +``` + +We can also subscribe to both events + +```python +# Subscribed to both events +@app.on('event1') +@app.on('event2') +def test5(message): + print("Function 5:", message) +``` + +```python +app.emit('event1', 'Hello, World!') +``` + +``` +Function 1: Hello, World! +Function 3: Hello, World! +Function 5: Hello, World! +``` + +```python +app.emit('event2', 'Goodbye, World!') +``` + +``` +Function 2: Goodbye, World! +Function 4: Goodbye, World! +Function 5: Goodbye, World! +``` + From cdc9540b1aff6a9d5b861fa634c61fab64a57658 Mon Sep 17 00:00:00 2001 From: Brandon Rozek Date: Sat, 18 Apr 2020 17:12:50 -0400 Subject: [PATCH 3/6] Added Post --- content/blog/internalca.md | 99 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100644 content/blog/internalca.md diff --git a/content/blog/internalca.md b/content/blog/internalca.md new file mode 100644 index 0000000..06b10ab --- /dev/null +++ b/content/blog/internalca.md @@ -0,0 +1,99 @@ +--- +title: "Quick CA for internal LAN" +date: 2020-04-18T16:26:53-04:00 +draft: false +tags: ["network"] +--- + +Setting up trusted HTTPs inside a network without exposure to the Internet requires creating a Certificate Authority. The audience for this post is oriented for people setting up services in a small low threat model environment. Additional cautions should be applied when setting this up for a business, for example working off an intermediate CA. + +We're going to be using [CFSSL](https://blog.cloudflare.com/introducing-cfssl/), this is Cloudflare's PKI toolkit to accomplish this. To install on Ubuntu, + +```bash +sudo apt install golang-cfssl +``` + +## Creating the CA + +This tool makes heavy use of JSON for its configuration. To setup a CA, first let's create `csr_ca.json` that contains the following information + +```json +{ + "CN": "Common Name", + "key": { + "algo": "rsa", + "size": 2048 + }, + "names": [ + { + "C": "US", + "O": "Orgnaization", + "OU": "Organizational Unit", + "ST": "Washington", + "L": "Locality" + } + ] +} +``` + +Where `C` is the two-letter country code and `ST` is the full state name. + +Then to create the certificate authority + +```bash +cfssl gencert -initca csr_ca.json | cfssljson -bare ca +``` + +This will create the following files + +| Filename | Purpose | +| ---------- | --------------------------- | +| ca.pem | Public Certificate | +| ca-key.pem | Private Key | +| ca.csr | Certificate Signing Request | + +## Creating Certficates + +Now we can create SSL certificates for whatever websites we wish by specifying in a file we'll call `csr_client.json` + +```json +{ + "hosts": [ + "example.com", + "*.example.com" + ], + "key": { + "algo": "rsa", + "size": 2048 + }, + "names": [ + { + "C": "US", + "O": "Orgnaization", + "OU": "Organizational Unit", + "ST": "Washington", + "L": "Locality" + } + ] +} +``` + +Then to create the certs, + +```bash +cfssl gencert -ca=ca.pem -ca-key=ca-key.pem csr_client.json | cfssljson -bare cert +``` + +It will create the private key, public certificate, and CSR just like the previous command. By default the certificate will last for one year and has the following usages: + +- Signing +- Key Encipherment +- Server Authentication +- Client Authentication + +To have more full grained control over the certificate usages and expiry time, I will defer you to the documentation. It involves creating another JSON file to pass as a flag into `cfssl gencert`. + +## Trusting the CA + +To trust the CA on Linux, you need to copy the `ca.pem` file over to `/usr/local/share/ca-certificates/` and then execute `sudo update-ca-certificates`. Firefox has its own certificate store that you can add `ca.pem` to by accessing Preferences->Privacy & Security->Security->Certificates->View Certificates->Authorities->Import. The exact trail might have changed by the time you read this. + From ecedff9c5549c18b56078b4e2db55dc0a209d7d8 Mon Sep 17 00:00:00 2001 From: Brandon Rozek Date: Sat, 18 Apr 2020 18:41:55 -0400 Subject: [PATCH 4/6] New posts --- content/blog/pycacheprop.md | 29 +++++++++++++++++++++++++++++ content/blog/quickpythonhttp.md | 14 ++++++++++++++ 2 files changed, 43 insertions(+) create mode 100644 content/blog/pycacheprop.md create mode 100644 content/blog/quickpythonhttp.md diff --git a/content/blog/pycacheprop.md b/content/blog/pycacheprop.md new file mode 100644 index 0000000..099e852 --- /dev/null +++ b/content/blog/pycacheprop.md @@ -0,0 +1,29 @@ +--- +title: "Quick Python: Cached Property" +date: 2020-04-18T18:29:21-04:00 +draft: false +tags: ["python"] +--- + +If you have a property in an object that only needs to be computed once, consider using `cached_property` to store the result and serve for future function calls. + +```python +import functools +class Number: + def __init__(self, n): + self.n = n + @functools.cached_property + def is_prime(self): + return all(self.n % i for i in range(2, self.n)) +``` + +Let's test it with the Mersenne prime `524287`. + +```python +n = Number(524287) +n.is_prime +``` + +After maybe 1-2 seconds of thinking you should get `True.` + +Run it again and the result will be instantaneous! \ No newline at end of file diff --git a/content/blog/quickpythonhttp.md b/content/blog/quickpythonhttp.md new file mode 100644 index 0000000..e23ae8f --- /dev/null +++ b/content/blog/quickpythonhttp.md @@ -0,0 +1,14 @@ +--- +title: "Quick Python: HTTP Server" +date: 2020-04-18T17:15:09-04:00 +draft: false +tags: [] +--- + +You can use Python to quickly spin up a HTTP server. A common use case for me is to quickly transfer files to mobile devices in my internal network. + +```python +python -m http.server +``` + +This will likely start an HTTP server on port 8000 on your machine listening to all network interfaces. \ No newline at end of file From 0228969117b60a51378724cacf6f2a6ee506fd84 Mon Sep 17 00:00:00 2001 From: Brandon Rozek Date: Mon, 20 Apr 2020 20:51:28 -0400 Subject: [PATCH 5/6] New Post --- content/blog/audioreplace.md | 55 ++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 content/blog/audioreplace.md diff --git a/content/blog/audioreplace.md b/content/blog/audioreplace.md new file mode 100644 index 0000000..0668267 --- /dev/null +++ b/content/blog/audioreplace.md @@ -0,0 +1,55 @@ +--- +title: "Replace Audio in Video" +date: 2020-04-20T20:32:26-04:00 +draft: false +tags: [] +--- + +I recorded a video and wanted to touch up my audio in audacity. Here's how I used `ffmpeg` to extract the audio, and then replace it with a modified version. + +## Extract Audio + +If you know the format of the audio (mp3, ogg, aac) then it's possible to do a byte copy of the audio track into a file: + +```bash +ffmpeg -i input_video.mkv -vn -acodec copy output.aac +``` + +| Argument | Description | +| -------------- | -------------------------- | +| `-i` | Input | +| `-vn` | No Video | +| `-acodec copy` | Copy audio stream directly | + +If you don't know the audio codec and have `mediainfo` installed, then run + +```bash +mediainfo --Inform="Audio;%Format%" input_video.mkv +``` + +If you gave up, then you can transcode the audio (will take longer than direct copy) + +```bash +ffmpeg -i input_video.mkv -vn output.aac +``` + +## Replacing Audio + +Once you're done touching up the audio (`touchup.mp3`), you'll want to replace the existing audio with it. + +```bash +ffmpeg -i input_video.mkv \ + -i touchup.mp3 \ + -c:v copy \ + -map 0:v:0 \ + -map 1:a:0 \ + output_video.mp4 +``` + +| Argument | Description | +| ---------------------- | ------------------------------------------------------------ | +| `-i` | Inputs | +| `-c:v copy` | Make this a copy operation | +| `-c:v copy -map 0:v:0` | Map the video from the first input to the first video output | +| `-map 1:a:0` | Map the audio from the second input to the first video output | + From 52c2b1edce7b31bfa11038bbefe7672fb9de7ca1 Mon Sep 17 00:00:00 2001 From: Brandon Rozek Date: Mon, 20 Apr 2020 21:12:43 -0400 Subject: [PATCH 6/6] Added audio only download method --- content/blog/youtubebackup.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/content/blog/youtubebackup.md b/content/blog/youtubebackup.md index f17d4f3..c1068fe 100644 --- a/content/blog/youtubebackup.md +++ b/content/blog/youtubebackup.md @@ -22,3 +22,19 @@ youtube-dl --ignore-errors \ URL_HERE ``` +## Audio Only + +To extract only audio here's the modified command + +```bash +youtube-dl --ignore-errors \ + --playlist-reverse \ + --output "%(uploader)s/%(uploader)s - %(title)s - %(upload_date)s.%(ext)s" \ + --format "bestvideo[ext=mp4]+bestaudio[ext=m4a]" \ + --merge-output-format mp4 \ + --embed-thumbnail \ + --add-metadata \ + --extract-audio + URL_HERE +``` +