Friday, 12 May 2023

Favourite .style.yapf

[style]
based_on_style = pep8
column_limit = 100
indent_width = 4
allow_split_before_dict_value = false
split_before_named_assigns = false
split_before_closing_bracket = true
split_before_first_argument = false
split_before_expression_after_opening_paren = false
split_arguments_when_comma_terminated = true
COALESCE_BRACKETS = true
DEDENT_CLOSING_BRACKETS = true
INDENT_CLOSING_BRACKETS = false
JOIN_MULTIPLE_LINES = true
SPLIT_ALL_TOP_LEVEL_COMMA_SEPARATED_VALUES = false
SPLIT_COMPLEX_COMPREHENSION = true
SPLIT_ALL_COMMA_SEPARATED_VALUES = false
EACH_DICT_ENTRY_ON_SEPARATE_LINE = false
BLANK_LINES_AROUND_TOP_LEVEL_DEFINITION = 1

Wednesday, 8 February 2023

PyCall in Julia to work on Windows with Anaconda

Julia ships its own Miniconda distro and uses to it by default. It actually tries to install this distro, and fails in my case. It fails to many other people too. It cannot know you already have another Python setup available. Help it with the following, when building PyCall. Note that you need to build PyCall manually before using it, or you'll get ERROR: LoadError: PyCall not properly installed. Please run Pkg.build("PyCall").

In Julia prompt, execute:

import Pkg
ENV["PYTHON"] = raw"C:\Users\YOUR_USER\anaconda3\python.exe"
Pkg.build("PyCall")

Tuesday, 7 February 2023

Jupyter Lab shortcut, dir listing in a current folder

Make a regular shortcut in a directory where you want Jupyter Lab to open with this target:

C:\WINDOWS\system32\cmd.exe /K "call %HOMEDRIVE%%HOMEPATH%\Anaconda3\Scripts\activate.bat & call jupyter lab && exit"

Set "Start in" as %cd%, and "Run" as Minimized. Done!




Saturday, 5 November 2022

Download entire website with wget or httrack

Before I used to run HTTrack (Windows, GUI version), but most of the time it wasn't what I wanted.

Here is a complete wget solution:

wget -r --no-parent --mirror --page-requisites --adjust-extension --convert-links --continue -e robots=off  https://www.website.com/

UPD: Yet, after a while, that was still isn't what I wanted :) I needed to crawl all pages which only belong to this website's domain PLUS the files like PDF the website references which are outside of site's domain. WGET won't give you this. HTTRACK in its CONSOLE incarnation, however, does. Here is the solution:

httrack --near -u2 -p7 https://YOUR_SITE

Note, it's GUI version won't do! Moreover, I found no switch in its GUI where these flags could be activated, same as no place to add cmd prompt flags manually.

Saturday, 29 October 2022

Download a youtube playlist

 Put ffmpeg.exe and yt-dlp (latest versions) to the folder where files to be downloaded.

The following command will download a playlist ${YOUR_PLAYLIST} in a subfolder with its reference name, enumerating the videos as they are ordered in the list, and assigning their names to the files. Numeration will start with 01 and with the oldest video. We need this to be able to keep the numeration later when we update the contents. The file archive.txt is needed to track the update and only download what hasn't been downloaded yet.

yt-dlp --format "bestvideo[height<=4k]+bestaudio/best" https://www.youtube.com/${YOUR_PLAYLIST} --embed-thumbnail --add-metadata --parse-metadata "description:(?s)(?P<webpage_url>.+)" --playlist-reverse --download-archive _archive --write-auto-sub --sub-langs "en,ru" -o "%%(playlist_index)s - %%(title)s - %%(id)s.%%(ext)s" --compat-option playlist-index