Skip to content

Instantly share code, notes, and snippets.

@benprew
Last active April 12, 2026 13:19
Show Gist options
  • Select an option

  • Save benprew/dcae81a3e56ed73c225aadedec397c90 to your computer and use it in GitHub Desktop.

Select an option

Save benprew/dcae81a3e56ed73c225aadedec397c90 to your computer and use it in GitHub Desktop.
A custom search engine to quickly access Recurse resources - design doc

RCLol: A custom search engine to quickly access Recurse resources

Quickly get to Recurse resources (calendar, github, wiki, etc).

We define keywords and their URL endpoints and then when you type them in your browser bar it takes you there! Ex. typing ‘cal’ into your browser would take you to the Recurse calendar. A visual example

Inspired by bunnylol at Meta. https://developers.facebook.com/blog/post/2020/06/03/build-smart-bookmarking-tool-rust-rocket/

Goals

V1: Quick Links

  • Keyword-based redirects to common Recurse resources (calendar, Zulip, wiki, etc.)
  • CRUD interface for managing keywords
  • Auto-redirect to a default search engine (e.g. Google) when keyword is not found
  • Low RAM usage
  • Parallelizeable - should be able to respond to multiple read/write requests at once
  • Reliable - system should be stable and require minimal maintenance/administration
  • Durable - data store/keywords should persist between app restarts and reboots

V2: Search and Aggregation

  • Full-text search across Recurse data (Zulip, calendar, community, posts, etc.)
  • Search result pages within the search engine
  • “Not found, did you mean X?” suggestion page
  • Auth and permissions for private data (Zulip messages, Recurser info)

System Design

Basic Flow

  1. User sets RCLol as custom search engine
  2. User types query into browser bar
  3. RCLol webserver receives request and does a lookup
    • If found, redirect to URL with arguments passed as query params
    • If not found:
      • V1: Auto-redirect to a default search engine with the search query
      • V2: Show “not found, did you mean X?” suggestion page

Keyword Format

Keywords support argument substitution, defined when creating the link.

  • %s is replaced with all arguments
  • %1, %2, etc. for positional arguments

Examples:

KeywordURL TemplateUsageResult
ggoogle.com/search?q=%sg recurse centergoogle.com/search?q=recurse+center
calrecurse.com/calendarcalrecurse.com/calendar
zzulip.example.com/#q=%sz machine learningzulip.example.com/#q=machine+learning

Backend

  • RCLol website (running on a local webserver)
    • Endpoint for receiving search queries
    • UI for managing keywords (CRUD interface)

Tech Stack

TBD depending on interested RCers. Could be JavaScript, Python, Ruby, Go, or even Zig. Mostly out of scope for this design doc.

Data Storage

Keyword mappings will live in a key value store (ex. Postres, MySQL, LMDB, Sqlite). Need a system that is durable, low resource usage, linux compatible and has good read performance. Writes are much less frequent than reads.

Keywords:

coltypedesc
keywordstringKeyword to type
dest_urlstringdestination URL (with placeholders)
created_attime
created_byintFK to users.id
updated_attime
updated_byintFK to user who most recently updated

Users - only if we can’t get from oauth:

coltypedesc
idserialuser id
namestringuser name

Deployment

  • Running on RC-local hardware (Heap cluster?)
  • Repo will be hosted on recurse org github
  • Deployment and installation TBD

Data Privacy

V2 search results involving Zulip messages and Recurser names/info will need auth and permissions.

V1 Default Links

  • library - www.libib.com/u/recursecenter
  • calendar - recurse.com/calendar
  • zulip - recurse.zulipchat.com
  • wiki - github.com/recursecenter/wiki/wiki
  • community/forum - community.recurse.com
  • phoneroom - phoneroom.recurse.com
  • rapidriter - rapidriter.rcdis.co
  • rcade - rcade.dev

V2: Search and Aggregation (Future Work)

Collect Recurse data and make it searchable. Results displayed as pages within RCLol.

  • Rcade info / list of Rcade games
  • Recurser names
  • Posts
  • Zulip messages
@benprew
Copy link
Copy Markdown
Author

benprew commented Apr 12, 2026 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment