Commit graph

1297 commits

Author SHA1 Message Date
Simon Sawicki
3ee1194288
[ie] Make _search_nextjs_data non fatal (#8937)
Authored by: Grub4K
2024-04-21 13:40:38 +02:00
Simon Sawicki
ff07792676
[core] Prevent RCE when using --exec with %q (CVE-2024-22423)
The shell escape function now properly escapes `%`, `\\` and `\n`. `utils.Popen` as well as `%q` output template expansion have been patched accordingly.

Prior to this fix using `--exec` together with `%q` when on Windows could cause remote code to execute. See https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-hjq6-52gw-2g7p for more details.

Authored by: Grub4K
2024-04-09 18:36:13 +02:00
bashonly
216f6a3cb5
[cleanup] Misc (#9426)
Authored by: bashonly, pukkandan
2024-04-09 16:12:26 +00:00
Simon Sawicki
32abfb00bd
[utils] traverse_obj: Convenience improvements (#9577)
Add support for:
- `http.cookies.Morsel`
- Multi type filters (`{type, type}`)

Authored by: Grub4K
2024-04-01 02:12:03 +02:00
pukkandan
c305a25c1b
[cleanup] Standardize import datetime as dt (#8978) 2024-04-01 05:32:15 +05:30
pukkandan
86a972033e
Infer acodec for single-codec containers 2024-03-31 22:50:21 +05:30
Simon Sawicki
3699eeb67c
[utils] traverse_obj: Allow unbranching using all and any (#9571)
Authored by: Grub4K
2024-03-30 19:54:43 +01:00
Simon Sawicki
979ce2e786
[test] traversal: Separate traversal tests (#9574)
Authored by: Grub4K
2024-03-30 19:32:07 +01:00
coletdjnz
52f5be1f1e
[rh:curlcffi] Add support for curl_cffi
Authored by: coletdjnz, Grub4K, pukkandan, bashonly

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: bashonly <bashonly@protonmail.com>
2024-03-16 23:15:11 -05:00
coletdjnz
0b81d4d252
Add new options --impersonate and --list-impersonate-targets
Authored by: coletdjnz, Grub4K, pukkandan, bashonly

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: bashonly <bashonly@protonmail.com>
2024-03-16 23:14:13 -05:00
coletdjnz
f849d77ab5
[test] Workaround websocket server hanging (#9467)
Authored by: coletdjnz
2024-03-16 16:57:21 +13:00
pukkandan
615a84447e
[cleanup] Misc (#8968)
Authored by: pukkandan, bashonly, seproDev
2024-03-11 00:52:28 +05:30
pukkandan
47ab66db0f
[docs] Misc Cleanup (#8977)
Closes #8355, #8944

Authored by: bashonly, Grub4k, Arthurszzz, seproDev, pukkandan

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
Co-authored-by: bashonly <bashonly@protonmail.com>
Co-authored-by: Arthurszzz <minecraftgamerarthur@gmail.com>
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-03-11 00:48:47 +05:30
sepro
ac340d0745
[test:websockets] Fix timeout test on Windows (#9344)
Authored by: seproDev
2024-03-04 17:47:38 +01:00
Lev
104a7b5a46
[ie] Migrate commonly plural fields to lists (#8917)
Authored by: llistochek, pukkandan
Related: #3944
2024-02-20 12:49:24 +05:30
coletdjnz
0085e2bab8
[rh] Remove additional logging handlers on close (#9032)
Fixes https://github.com/yt-dlp/yt-dlp/issues/8922

Authored by: coletdjnz
2024-02-18 11:32:34 +13:00
coletdjnz
35f4f764a7
[rh:requests] Apply remove_dot_segments to absolute redirect locations
Fixes https://github.com/yt-dlp/yt-dlp/issues/9020

Authored by: coletdjnz
2024-01-21 10:03:33 +13:00
coletdjnz
811d298b23
[networking] Remove _CompatHTTPError (#8871)
Use `yt_dlp.networking.exceptions.HTTPError`.
`_CompatHTTPError` was to help with transition to the networking framework.

Authored by: coletdjnz
2024-01-20 15:26:50 +13:00
coletdjnz
69d3191495
[test] Skip source address tests if the address cannot be bound to (#8900)
Fixes https://github.com/yt-dlp/yt-dlp/issues/8890

Authored by: coletdjnz
2024-01-20 10:39:49 +13:00
Simon Sawicki
ffbd4f2a02
[utils] traverse_obj: Support xml.etree.ElementTree.Element (#8911)
Authored by: Grub4K
2024-01-05 21:26:17 +01:00
Simon Sawicki
f9fb3ce86e
[cleanup] Misc (#8598)
Authored by: bashonly, pukkandan, seproDev, Grub4K

Co-authored-by: bashonly <bashonly@protonmail.com>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2023-12-30 22:27:36 +01:00
Simon Sawicki
00cdda4f6f
[core] Fix format selection parse error for CPython 3.12 (#8797)
Authored by: Grub4K
2023-12-24 22:09:01 +01:00
bashonly
632b8ee54e
[core] Release workflow and Updater cleanup (#8640)
- Only use trusted publishing with PyPI and remove support for PyPI tokens from release workflow
- Clean up improper actions syntax in the build workflow inputs
- Refactor Updater to allow for consistent unit testing with `UPDATE_SOURCES`

Authored by: bashonly
2023-12-21 21:06:26 +00:00
bashonly
37755a037e
[test:networking] Update tests for OpenSSL 3.2 (#8814)
Authored by: bashonly
2023-12-20 19:03:54 +00:00
coletdjnz
196eb0fe77
[networking] Strip whitespace around header values (#8802)
Fixes https://github.com/yt-dlp/yt-dlp/issues/8729
Authored by: coletdjnz
2023-12-20 19:15:38 +13:00
Simon Sawicki
0b6f829b1d
[utils] traverse_obj: Move is_user_input into output template (#8673)
Authored by: Grub4K
2023-12-06 21:46:45 +01:00
pukkandan
993edd3f6e
[outtmpl] Support multiplication
Related: #8683
2023-12-06 03:44:11 +05:30
sepro
1732eccc0a
[core] Parse release_year from release_date (#8524)
Closes #7263
Authored by: seproDev
2023-11-26 02:12:05 +00:00
coletdjnz
ccfd70f4c2
[rh:websockets] Migrate websockets to networking framework (#7720)
* Adds a basic WebSocket framework
* Introduces new minimum `websockets` version of 12.0
* Deprecates `WebSocketsWrapper`

Fixes https://github.com/yt-dlp/yt-dlp/issues/8439

Authored by: coletdjnz
2023-11-20 08:04:04 +00:00
sepro
45d82be65f
[ie/nebula] Overhaul extractors (#8566)
Closes #4300, Closes #5814, Closes #7588, Closes #6334, Closes #6538
Authored by: elyse0, pukkandan, seproDev

Co-authored-by: Elyse <26639800+elyse0@users.noreply.github.com>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2023-11-20 01:03:33 +00:00
bashonly
f4b95acafc
Remove Python 3.7 support (#8361)
Closes #7803
Authored by: bashonly
2023-11-16 18:39:00 +00:00
bashonly
f04b5bedad
[ie] Do not smuggle http_headers
See: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-3ch3-jhc6-5r8x

Authored by: coletdjnz
2023-11-14 22:04:25 +01:00
bashonly
87264d4fda
[test:update] Implement simple updater unit tests
Authored by: bashonly
2023-11-12 18:30:55 -06:00
Simon Sawicki
4e38e2ae9d
[rh:requests] Handle both bytes and int for IncompleteRead.partial (Fix 8a8b54523a) (#8348)
Authored by: bashonly, coletdjnz, Grub4K
2023-10-15 10:54:38 +02:00
coletdjnz
8a8b54523a
[rh:requests] Add handler for requests HTTP library (#3668)
Adds support for HTTPS proxies and persistent connections (keep-alive)

Closes https://github.com/yt-dlp/yt-dlp/issues/1890
Resolves https://github.com/yt-dlp/yt-dlp/issues/4070
Resolves https://github.com/ytdl-org/youtube-dl/issues/32549
Resolves https://github.com/ytdl-org/youtube-dl/issues/14523
Resolves https://github.com/ytdl-org/youtube-dl/issues/13734

Authored by: coletdjnz, Grub4K, bashonly
2023-10-13 23:33:00 +00:00
Awal Garg
9d7ded6419
[utils] js_to_json: Fix Date constructor parsing (#8295)
Authored by: awalgarg, Grub4K
2023-10-08 01:57:23 +02:00
Simon Sawicki
088add9567
[cleanup] Misc
Authored by: Grub4K
2023-09-24 02:35:23 +02:00
Simon Sawicki
de015e9307
[core] Prevent RCE when using --exec with %q (CVE-2023-40581)
The shell escape function is now using `""` instead of `\"`. `utils.Popen` has been patched to properly quote commands.

Prior to this fix using `--exec` together with `%q` when on Windows could cause remote code to execute. See https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-42h4-v29r-42qg for reference.

Authored by: Grub4K
2023-09-24 02:29:01 +02:00
Simon Sawicki
61bdf15fc7
[core] Raise minimum recommended Python version to 3.8 (#8183)
Authored by: Grub4K
2023-09-24 02:24:47 +02:00
bashonly
5ca095cbcd
[cleanup] Misc (#8182)
Closes #7796, Closes #8028
Authored by: barsnick, sqrtNOT, gamer191, coletdjnz, Grub4K, bashonly
2023-09-23 20:00:31 +00:00
bashonly
661c9a1d02
[test:download] Test for expected_exception
Authored by: at-wat

Co-authored-by: Atsushi Watanabe <atsushi.w@ieee.org>
2023-09-21 17:48:57 -05:00
bashonly
52414d64ca
[utils] js_to_json: Handle Array objects
Authored by: Grub4K, std-move

Co-authored-by: std-move <26625259+std-move@users.noreply.github.com>
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
2023-09-21 16:51:57 -05:00
coletdjnz
20fbbd9249
[networking] Fix various socks proxy bugs (#8065)
- Fixed support for IPv6 socks proxies
- Fixed support for IPv6 over socks5
- Fixed --source-address not being obeyed for socks4 and socks5
- Fixed socks4a when the destination address is an IPv4 address

Closes https://github.com/yt-dlp/yt-dlp/issues/7959
Fixes https://github.com/ytdl-org/youtube-dl/issues/15368

Authored by: coletdjnz
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
Co-authored-by: bashonly <bashonly@bashonly.com>
2023-09-18 07:33:26 +00:00
coletdjnz
fcd6a76adc
[tests] Add tests for socks proxies (#7908)
Authored by: coletdjnz
2023-08-25 07:10:44 +00:00
coletdjnz
db7b054a61
[networking] Add request handler preference framework (#7603)
Preference functions that take a request and a request handler instance can be registered to prioritize different request handlers per request.

Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2023-08-04 22:17:48 +00:00
pukkandan
6148833f5c
[cleanup] Misc 2023-07-30 04:06:18 +05:30
pukkandan
3f7965105d
[utils] HTTPHeaderDict: Handle byte values 2023-07-30 03:18:10 +05:30
pukkandan
de20687ee6
[test] Fix test_load_certifi
Closes #7688, #7675
2023-07-29 21:53:00 +05:30
coletdjnz
4bf912282a
[networking] Remove dot segments during URL normalization (#7662)
This implements RFC3986 5.2.4 remove_dot_segments during the URL normalization process.

Closes #3355, #6526

Authored by: coletdjnz
2023-07-28 22:40:20 +00:00
coletdjnz
bbeacff7fc
[networking] Ignore invalid proxies in env (#7704)
Authored by: coletdjnz
2023-07-27 20:26:02 +05:30