This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# Script to simplify the release flow. | |
# 1) Fetch the current release version | |
# 2) Increase the version (major, minor, patch) | |
# 3) Add a new git tag | |
# 4) Push the tag | |
# Parse command line options. | |
while getopts ":Mmpd" Option |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
FROM gradle:6.3-jdk11 | |
# Install Docker client | |
ARG DOCKER_VERSION=19.03.8 | |
RUN curl -fsSL https://download.docker.com/linux/static/stable/`uname -m`/docker-$DOCKER_VERSION.tgz | tar --strip-components=1 -xz -C /usr/local/bin docker/docker | |
# Install Kubectl | |
RUN curl -LfsO https://storage.googleapis.com/kubernetes-release/release/v1.18.0/bin/linux/amd64/kubectl \ | |
&& chmod +x ./kubectl \ | |
&& mv ./kubectl /usr/local/bin/kubectl |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Markus Bash Prompt, inspired by "Sexy Prompt" | |
if tput setaf 1 &> /dev/null; then | |
if [[ $(tput colors) -ge 256 ]] 2>/dev/null; then | |
MAGENTA=$(tput setaf 9) | |
ORANGE=$(tput setaf 172) | |
GREEN=$(tput setaf 190) | |
PURPLE=$(tput setaf 141) | |
WHITE=$(tput setaf 0) | |
else |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
version: '3' | |
services: | |
zeppelin: | |
build: ./zeppelin/docker | |
image: zeppelin:latest | |
ports: | |
- 10000:8080 | |
volumes: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
var account = spark.read | |
.format("jdbc") | |
.option("url", "jdbc:postgresql://localhost:5432/zeppelin") | |
.option("dbtable", "public.account") | |
.option("user", "zeppelin") | |
.option("password", "zeppelin") | |
.option("driver", "org.postgresql.Driver") | |
.load() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
account.write | |
.format("jdbc") | |
.option("url", "jdbc:postgresql://localhost:5432/zeppelin") | |
.option("dbtable", "public.account") | |
.option("user", "zeppelin") | |
.option("password", "zeppelin") | |
.option("driver", "org.postgresql.Driver") | |
.mode("overwrite") | |
.save |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val filesPath = "/Volumes/Volume/bank-account-files/*.csv" | |
import org.apache.spark.sql.types.{StructType, StructField, StringType, DecimalType}; | |
val customSchema = StructType( Array( StructField("Buchung", StringType, true), | |
StructField("Wert", StringType, true), | |
StructField("Verwendungszweck", StringType, true), | |
StructField("Betrag", StringType, true) )) | |
val df = sqlContext.read.format("com.databricks.spark.csv"). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# | |
# There is insufficient memory for the Java Runtime Environment to continue. | |
# Native memory allocation (mmap) failed to map 715915264 bytes for committing reserved memory. | |
# Possible reasons: | |
# The system is out of physical RAM or swap space | |
# In 32 bit mode, the process size limit was hit | |
# Possible solutions: | |
# Reduce memory load on the system | |
# Increase physical memory or swap space | |
# Check if swap backing store is full |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
.DS_Store | |
.DocumentRevisions-V100 | |
.Spotlight-V100 | |
.TemporaryItems | |
.Trashes | |
.com.apple.timemachine.donotpresent | |
.fseventsd | |
_gsdata_ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
rsync -avzi \ | |
--delete --delete-excluded --force-delete --progress \ | |
--exclude-from /Volumes/MyData/backup-excluded.txt \ | |
/Volumes/MyData/ /Volumes/Backup/MyData/ |
NewerOlder