seanorama

PySpark on YARN in self-contained environments

Author: https://github.com/seanorama

Note: This was tested on HDP 3.1. It may not work with other Spark/YARN distributions.

Reference:

• https://community.cloudera.com/t5/Community-Articles/Using-VirtualEnv-with-PySpark/ta-p/245905
• https://community.cloudera.com/t5/Community-Articles/Running-PySpark-with-Conda-Env/ta-p/247551

fearblackcat

Shadowsocks Proxy

Server-side setup

apt-get install python-pip
pip install shadowsocks

sudo ssserver -p 443 -k password -m aes-256-cfb --user nobody -d start

nicolasembleton

#!/bin/bash

sudo kextunload -b com.apple.iokit.BroadcomBluetoothHostControllerUSBTransport
sudo kextload -b com.apple.iokit.BroadcomBluetoothHostControllerUSBTransport