nokados’s gists

nokados / gdrive.py

Created February 28, 2019 00:34

Download large files from Google Drive

	import requests

	def download_file_from_google_drive(id, destination):
	def get_confirm_token(response):
	for key, value in response.cookies.items():
	if key.startswith('download_warning'):
	return value

	return None

nokados / multilabel_stratified_train_test_split.py

Last active March 8, 2019 14:57

Analogue of sklearn's train_test_split for multilabel classification with stratification and shuffling. And also under/over sampling to make distributions of class lengths more flat.

	import numpy as np
	import matplotlib.pyplot as plt
	%matplotlib inline

	def parallel_shuffle(*arrays):
	length = arrays[0].shape[0]
	for arr in arrays:
	assert arr.shape[0] == length
	p = np.random.permutation(length)

nokados / jupyter.service

Last active March 31, 2020 12:48 — forked from whophil/jupyter.service

A systemd script for running a Jupyter notebook server.

	# After Ubuntu 16.04, Systemd becomes the default.
	# It is simpler than https://gist.github.com/Doowon/38910829898a6624ce4ed554f082c4dd

	[Unit]
	Description=Jupyter Notebook

	[Service]
	Type=simple
	PIDFile=/run/jupyter.pid
	ExecStart=/home/nokados/anaconda3/bin/jupyter-notebook --config=/home/nokados/.jupyter/jupyter_notebook_config.py --no-browser

nokados / pandas_jupyter_paginator.py

Last active June 17, 2022 04:23

Paginator for pandas.DataFrame in Jupyter Notebook. UPD: use https://github.com/nvictus/pandas-jupyter-paginate instead

	from IPython.core.display import display, HTML, clear_output
	from ipywidgets import widgets, Layout
	import math
	PAGESIZE=10

	class Paginator:
	def __init__(self, df, pagesize = PAGESIZE, start_page = 0):
	self.df = df
	self.pagesize = pagesize
	self.page = start_page

nokados / parse_article.py

Last active October 22, 2018 16:04

	import json, requests, time
	import datetime
	import pickle
	from collections import Counter
	from pyquery import PyQuery
	from readability import Document
	from lxml.etree import XMLSyntaxError, LxmlError
	from readability.readability import Unparseable
	from requests.adapters import MaxRetryError
	from requests.exceptions import ConnectionError

nokados / bind.py

Created August 27, 2018 12:06

method that allows to bind some function or method to the current class

	def bind(self, method):
	def new_method(new_self, args, *kwargs):
	new_self.data = method(self.data, args, *kwargs)
	return new_self
	setattr(self, method.__name__, types.MethodType(new_method, self))

nokados / translit.py

Last active June 12, 2018 22:20 — forked from ledovsky/translit.py

Транслитерация на python

	# name: это строка которую транслитим
	def transliterate(name):
	"""
	Автор: LarsKort
	Дата: 16/07/2011; 1:05 GMT-4;
	Не претендую на "хорошесть" словарика. В моем случае и такой пойдет,
	вы всегда сможете добавить свои символы и даже слова. Только
	это нужно делать в обоих списках, иначе будет ошибка.
	"""
	# Слоаврь с заменами

nokados / has.py

Created June 3, 2018 20:56

Closure. Check if a string includes a pattern.

	def has(expr):
	return lambda x: bool(re.search(expr, x, flags=re.IGNORECASE\|re.MULTILINE\|re.DOTALL))

nokados / init_jupyter.py

Last active July 15, 2018 12:05

Snippet for the first block in Jupyter Notebook. Import core libraries, set random seed, enable autoreload imported files

	%load_ext autoreload
	%autoreload 2

	import pandas as pd
	from tqdm import tqdm_notebook, tqdm_pandas, tnrange
	import time
	import numpy as np
	from IPython.display import clear_output
	import pickle as pkl
	import os

nokados / keras_word2vec_embedding.py

Created May 25, 2018 09:59

Embedding Layer for Keras with weights from gensim Word2Vec (or FastText, why not?)

	def word2vec_embedding_layer(embeddings_path='data/weights.npz', max_review_length=150):
	weights = load_weights(embeddings_path)
	layer = Embedding(input_dim=weights.shape[0],
	output_dim=weights.shape[1],
	input_length=max_review_length,
	weights=[weights])
	return layer

	# How to make 'data/weights.npz'? What is load_weights()?
	# See https://gist.github.com/nokados/d5cfec00bc194822f89dff556ff62b29

Nikita Furin nokados