Skip to content

Instantly share code, notes, and snippets.

View johnnymo87's full-sized avatar

Jonathan Mohrbacher johnnymo87

View GitHub Profile
@johnnymo87
johnnymo87 / print-offsets.bash
Last active September 18, 2024 20:57
Kafka Partition End Offset Checker
#!/usr/bin/env bash
# Kafka Partition End Offset Checker
# ==================================
#
# This script queries a Kafka cluster to retrieve the end offset information for all partitions
# of a specified topic. It provides the log end offset for each partition, which represents
# the offset where the next message would be appended.
#
# Usage:
@johnnymo87
johnnymo87 / guide.md
Created August 13, 2024 19:43
Setting up a Ruby on Rails app on Mac OS X
@johnnymo87
johnnymo87 / Skipping-kafka-messages.md
Last active August 9, 2024 10:02
A guide about using the `kafka-consumer-groups` CLI to skip stuck kafka messages

What's a stuck message?

FES consumes events from the fes-async-in and the sts-delivery-tracking-events kafka event streams. The latter has never posed a problem, so this document is geared to talk about the former. Regarding fes-async-in, our order management systems send and receive kafka messages through it to communicate with our fulfillment management system. Prior to the solution, we in FMS would occasionally receive a kafka message that we couldn’t consume due to a bug, bad data, etc. When this happened, our kafka consumer tried this message again and again, literally forever. This “just try again” policy is great for transient errors, such as database locking. However, it's a problem if this error isn’t transient, as all other messages behind it are now blocked and will never be processed. This is what we call a "stuck message".

Inspired by this Uber blog post, we wrote [special "punt" code](https://github.com/blueapron/fulfillment-engine/pull

@johnnymo87
johnnymo87 / code_concatenator.bash
Last active May 11, 2024 21:14
Concatenates code files from a directory and its subdirectories into a single output file.
#!/usr/bin/env bash
: <<'END'
Script Name: code_concatenator.bash
Purpose:
This script is designed to concatenate all code files within a specified directory
and its subdirectories into a single output file. The output file will contain the
file paths and contents of each code file, separated by a delimiter (```) This
script is particularly useful for preparing code files for analysis or processing
@johnnymo87
johnnymo87 / compress_photos.bash
Last active April 16, 2024 13:10
Photo compression script to compress directory images under 25MB total
#!/usr/bin/env bash
: <<'END'
Script Name: compress_photos.bash
Purpose:
This script is designed to compress a directory of photos to ensure the total
size is under a specified limit (25 MB by default). It's particularly useful
for preparing image files for platforms with strict upload size limits or for
optimizing storage. The script iterates over a set of image files, applying
@johnnymo87
johnnymo87 / set-read-only.bash
Created February 27, 2024 21:51
Using the GitHub graphql API, set the read-only setting on an existing protection rule for a branch. When true, no one can merge pull requests to it.
#! /usr/bin/env bash
# Using the GitHub graphql API, set the read-only setting on an existing
# protection rule for a branch. When true, no one can merge pull requests to
# it.
#
# Dependencies:
# * gh (https://cli.github.com/)
# * A fine-grained GitHub personal access token, set as `GH_TOKEN` in the
# environment.
@johnnymo87
johnnymo87 / explanation.md
Last active December 22, 2023 16:52
Here's the fastest way to get raw SQL into an API endpoint, with minimal intervention from the model layer of the Rails framework.

To distill the essence of turning a raw SQL DB query into an API endpoint in Rails, we can abstract away the domain-specific details and focus on the general process. Here's a step-by-step guide that captures the technique:

Step 1: Define the SQL Query

Create a module with a method that encapsulates the raw SQL query. This method should accept parameters to filter or manipulate the data as needed and return the result set.

module Queries
  module CustomData
    def self.perform(params:)
 query = &lt;&lt;-SQL
number name
0 Black
1 Maroon
2 Green
3 Olive
4 Navy
5 Purple
6 Teal
7 Silver
8 Grey
@johnnymo87
johnnymo87 / plan.md
Last active April 24, 2023 13:52
How I pascal-cased a bunch screaming snake case variables
  • I am a neovim user and software engineer working on a rails 6.1 ruby 2.7 application.

  • All constants in the application use the SCREAMING_SNAKE_CASE naming convention.

  • The rubocop style guide suggests that I don't have to follow this convention when the constant refers to a class. For those that refer to a class, they can be PascalCased like classes normally are in ruby.

    image
  • I'm using ripgrep, and in vim I have a plugin where I can send search queries to ripgrep via the :Rg command.

  • I've written a ripgrep query to find all instances of where a class is assigned to a SCREAMING_SNAKE_CASE constant.

:Rg "[A-Z][A-Z0-9_]+\s+=\s+[A-Z][a-z]" -g "**/*.rb"

@johnnymo87
johnnymo87 / flip_flop_yard_doc.vim
Last active April 22, 2023 18:06
Vim script that I used to flip YARDoc from Java style to Ruby style
" Fix YARD documentation comments in one file.
function! FixYardComments()
" Search for incorrect @param syntax.
let search_pattern = '@param \[\(.*\)]\s\+\(\w\+\)'
" Replace incorrect syntax with correct one.
let replace_pattern = '@param \2 \[\1\]'
" Apply the replacement to the entire file.
execute '%s/' . search_pattern . '/' . replace_pattern . '/g'