Skip to content

Instantly share code, notes, and snippets.

View mithunkamat's full-sized avatar
💭
I may be slow to respond.

M_Dev mithunkamat

💭
I may be slow to respond.
View GitHub Profile
@mithunkamat
mithunkamat / rename_cols.py
Created May 18, 2020 07:10
Example of renaming columns using a 'mappings' dictionary which is dynamically built from a 'mapping' table from within an Excel sheet.
data = pd.read_csv(mappings['Filepath'])
# lowercase and strip whitespace from all column headers
data.rename({col: col.strip().lower() for col in data.columns},
axis=1, inplace=True)
# we must also do the same for inv_mappings
# also add snake_case formatting to internal columns (optional)
inv_mappings = {
mappings[key].strip().lower():
@mithunkamat
mithunkamat / search_mappings.py
Created May 18, 2020 07:10
Example code iteratively building a 'mappings' dictionary from a table in Excel using openpyxl. On seeing two blank lines, we assume the table has ended.
empty = 0 # initialize our empty cell count
while empty < 2:
# increment the row number (which begins at the row we found 'Internal'
row += 1
# assign the value we find in the 'Internal' column to internal
internal = ws[f'B{row}'].value
if internal is None:
empty += 1 # if we get blank row, increment empty counter
@mithunkamat
mithunkamat / search_col_func.py
Created May 18, 2020 07:10
Searching a column with openpyxl.
# define function to search a given column in an openpyxl worksheet object
def search_col(sheet, column, value, limit=100):
# loop through each row upto the limit, beginning from 1
for row in range(1, limit+1):
if sheet[f"column{row}"].value == value:
# if we find the value we are searching for, return col + row
return (col, row)
@mithunkamat
mithunkamat / get_mapping_tab.py
Created May 18, 2020 07:10
Snippet showing how to set a tab called 'Mapping' as the active sheet with openpyxl.
# first we set the path to Excel file
path = r".\documents\tool_setup.xlsx"
# open the file as a workbook object
wb = openpyxl.load_workbook(path, data_only=True)
# find the index of 'Mapping' sheet
idx = [i for i, name in wb.sheetnames if name == 'Mapping'][0]
# set the mapping sheet as active
@mithunkamat
mithunkamat / get_cell_value.py
Created May 18, 2020 07:09
Pulling a value from an Excel cell example.
import openpyxl
# loan the workbook object using 'sheet.xlsx'
wb = openpyxl.load_workbook("sheet.xlsx", data_only=True)
# create a worksheet object with the active sheet
ws = wb.active
# get the value in cell 'E4'
value = ws['E4'].value
@mithunkamat
mithunkamat / manual_mappings.py
Created May 18, 2020 07:09
Snippet demonstrating Pandas dataframe column renaming function.
mappings = {'loan identifier': 'loan_id',
'amt': 'amount',
...
'init fees': 'initial_fees'}
data.rename(mappings, axis=1, inplace=True)
# now, for the remainder of the code we use the data's Python given column names, 'loan_id', 'loan_amount', etc
@mithunkamat
mithunkamat / notify_message.py
Created May 18, 2020 07:07 — forked from jamescalam/notify_message.py
Example of using Python's email module to build an email message object containing a subject, body, attachments, and images.
import os
from email.mime.text import MIMEText
from email.mime.image import MIMEImage
from email.mime.application import MIMEApplication
from email.mime.multipart import MIMEMultipart
def message(subject="Python Notification", text="", img=None, attachment=None):
# build message contents
msg = MIMEMultipart()
msg['Subject'] = subject # add in the subject
@mithunkamat
mithunkamat / notify_model.py
Created May 18, 2020 07:06 — forked from jamescalam/notify_model.py
Example code using sending email notification on completion of a cash-flow model process.
end = datetime.datetime.now() # get the ending datetime
# get the total runtime in hours:minutes:seconds
hours, rem = divmod((end - start).seconds, 3600)
mins, secs = divmod(rem, 60)
runtime = '{:02d}:{:02d}:{:02d}'.format(hours, mins, secs)
# now built our message
notify.msg(
subject="Cashflow Model Completion",
@mithunkamat
mithunkamat / notify_sql.py
Last active May 18, 2020 07:06 — forked from jamescalam/notify_sql.py
Example usage for sending mobile notification upon completion of a data upload to MS SQL Server.
import os
import notify
from data import Sql # see https://jamescalam.github.io/pysqlplus/lib/data/sql.html
dt = Sql('database123', 'server001') # setup the connection to SQL Server
for i, file in enumerate(os.listdir('../data/new')):
dt.push_raw(f'../data/new/{file}') # push a file to SQL Server
# once the upload is complete, send a notification
@mithunkamat
mithunkamat / notify_ml.py
Created May 18, 2020 07:05 — forked from jamescalam/notify_ml.py
Example usage of notify for ML model training updates.
import notify
START = datetime.now() # this line would be placed before model training begins
MODELNAME = "Synthwave GAN" # giving us our model name
NOTIFY = 100 # so we send an update notification every 100 epochs
# for each epoch e, we would include the following code
if e % notify_epoch == 0 and e != 0:
# here we create the email body message
txt = (f"{MODELNAME} update as of "