Zachary King zcking

Data Architect with interests in cloud architecture, devops, data engineering, cloud native, and cutting edge technology.

zcking / spark_tle_data_source.py

Created April 13, 2025 18:17

Example of a custom pyspark data source for ingesting satellite TLE data and visualizing it with matplotlib.

	import matplotlib
	import pandas as pd
	import matplotlib.pyplot as plt
	import numpy as np
	from pyspark.sql.datasource import DataSource, DataSourceStreamReader, DataSourceReader, InputPartition
	from pyspark.sql.types import StructType
	from pyspark.sql import SparkSession, DataFrame
	import matplotlib.cm as cm

	import requests