sequenceDiagram
participant User as Intent Recognition
participant Controller
participant AutoML as AutoML Engine
participant Tabular as Tabular Engine
flowchart LR
%% Intent and Control
A[Intent Recognition] -->|Extracts task type & user input| B[Controller]
B -->|Requests requirements| C[AutoML Engine]
C -->|Returns requirements JSON| B
B -->|Sends config, data, task info| C
%% Core AutoML Tasks
subgraph Core_AutoML_Tasks["Core AutoML Tasks"]
- Subhaditya Mukhejee : [email protected]
I am Subhaditya, it's nice to meet you!
Below is a list of projects I have worked on for various clients over the years.
I wanted to keep a list of them for both myself and future employers.
That being said, I am almost always open for more freelancw work, so hit me up if you want to discuss some projects :)
- While most of the code is closed source due to client requirements, I am still open to discussing them to a certain extent
- My Github is https://github.com/SubhadityaMukherjee
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Run result aggregator | |
- Read all tensorbord logs and save as a pandas dataframe for analysis | |
""" | |
import os | |
import pandas as pd | |
from tensorboard.backend.event_processing.event_accumulator import EventAccumulator | |
from tqdm import tqdm | |
from pathlib import Path |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
filtered_df = combined_df[(~pd.isnull(combined_df['converted_proxy'])) & (~pd.isnull(combined_df['original_images']))] | |
filtered_df.iloc[0].original_images |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
combined_df = process_runs(main_path=main_path) | |
combined_df = combined_df[(~pd.isnull(combined_df['experiment_name'])) & (~pd.isnull(combined_df['Loss/Val']))] | |
combined_df.head() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import pickle | |
with open("pickled_df.pkl", "wb+") as f: | |
pickle.dump(combined_df, f) | |
with open("pickled_df.pkl", "rb+") as f: | |
combined_df = pickle.load(f) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def process_event_acc(event_acc, save_ims=False) -> dict: | |
"""Process the EventAccumulator and return a dictionary of tag values""" | |
all_tags = event_acc.Tags() # Get all tags | |
temp_dict = {} # Store all values here | |
for tag in all_tags.keys(): # Loop over all tags | |
if tag == "scalars": | |
# Process scalars | |
for subtag in all_tags[tag]: | |
try: | |
# Try to get the last value |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def process_runs(main_path, save_ims=False) -> pd.DataFrame: | |
"""Process all runs and return a dataframe of all results""" | |
all_files = get_event_files(main_path=main_path, save_ims=False) | |
all_dict = {} | |
for files in tqdm(all_files, total=len(all_files)): | |
try: | |
# Process each file using the EventAccumulator and save to a dictionary | |
event_acc = EventAccumulator(files) | |
event_acc.Reload() | |
temp_dict = process_event_acc(event_acc, save_ims=save_ims) |
NewerOlder