franchb

Try to implement https://github.com/wannesm/dtaidistance in Go, Rust, Zig just for learning some new stuff.

franchb

Optimizing postgresql table for more than 100K inserts per second

• Create UNLOGGED table. This reduces the amount of data written to persistent storage by up to 2x.
• Set WITH (autovacuum_enabled=false) on the table. This saves CPU time and IO bandwidth on useless vacuuming of the table (since we never DELETE or UPDATE the table).
• Insert rows with COPY FROM STDIN. This is the fastest possible approach to insert rows into table.
• Minimize the number of indexes in the table, since they slow down inserts. Usually an index on time timestamp with time zone is enough.
• Add synchronous_commit = off to postgresql.conf.
• Use table inheritance for fast removal of old data:

franchb

Run Docker commands without sudo

1. Add the docker group if it doesn't already exist

$ sudo groupadd docker

2. Add the connected user $USER to the docker group

franchb

RUN apk --no-cache add  \
        wget            \
        ca-certificates \
        libstdc++
# Get and install glibc for alpine
ARG APK_GLIBC_VERSION=2.29-r0
ARG APK_GLIBC_FILE="glibc-${APK_GLIBC_VERSION}.apk"
ARG APK_GLIBC_BIN_FILE="glibc-bin-${APK_GLIBC_VERSION}.apk"
ARG APK_GLIBC_BASE_URL="https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${APK_GLIBC_VERSION}"
RUN wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub \

franchb

My bash hacks

Soon I plan to move it to my own Ubuntu config.

1. cargo install exa && alias ll="exa -l" && alias llf="exa -alF" - pretty and faster ls alternative
2. https://github.com/cantino/mcfly for improved bash history
3. source /home/<user>/.config/broot/launcher/bash/br
4. https://github.com/junegunn/fzf bash fuzzy finder
5. https://github.com/denisidoro/navi bash interactive cheatsheet

franchb

ssh-keygen
	-t ed25519 - for greatest security (bits are a fixed size and -b flag will be ignored)
	-t rsa  - for greatest portability (key needs to be greater than 4096 bits)
	-t ecdsa - faster than RSA or DSA  (bits can only be 256, 284, or 521)
	-t dsa  - DEEMED INSECURE - DSA limted to 1024 bit key as specified by FIPS 186-2, No longer allowed by default in OpenSSH 7.0+
	-t rsa1 - DEEMED INSECURE - has weaknesses and shouldn't be used (used in protocol 1)

	-b 4096 bit size
	-a 500 rounds (should be no smaller than 64, result in slower passphrase verification and increased resistance to brute-force password cracking)
	-C "[email protected]" comment..

franchb

To remove a submodule you need to:

• Delete the relevant section from the .gitmodules file.
• Stage the .gitmodules changes git add .gitmodules
• Delete the relevant section from .git/config.
• Run git rm --cached path_to_submodule (no trailing slash).
• Run rm -rf .git/modules/path_to_submodule (no trailing slash).
• Commit git commit -m "Removed submodule "
• Delete the now untracked submodule files rm -rf path_to_submodule

franchb

#!/usr/bin/env python
#title           :distribution_checkX.py
#description     :Checks a sample against 80 distributions by applying the Kolmogorov-Smirnov test.
#author          :Andre Dietrich
#email           :[email protected]
#date            :07.10.2014
#version         :0.1
#usage           :python distribution_check.py -f filename -v
#python_version  :2.* and 3.*
#########################################################################################

franchb

# separation plot
# Author: Cameron Davidson-Pilon,2013
# see http://mdwardlab.com/sites/default/files/GreenhillWardSacks.pdf


import matplotlib.pyplot as plt
import numpy as np

franchb

# Original -- https://www.kaggle.com/pavansanagapati/weight-of-evidence-woe-information-value-iv


import pandas as pd
import numpy as np
import pandas.core.algorithms as algos
from pandas import Series
import scipy.stats.stats as stats
import re
import traceback