Skip to content

Instantly share code, notes, and snippets.

@dermidgen

dermidgen/VIAM-PoC-Notes.md

Last active April 11, 2025 16:08
Show Gist options
  • Save dermidgen/95ac6d627ebf47cfeaaaf9fd6080bf52 to your computer and use it in GitHub Desktop.
Save dermidgen/95ac6d627ebf47cfeaaaf9fd6080bf52 to your computer and use it in GitHub Desktop.
Download ZIP
Raw notes on the VIAM PoC assignment
Raw
VIAM-PoC-Notes.md

VIAM Project Notes

Session 1 (the accidental dive in): Monday April 7th, Start 1725 | End 1755

It’s late in the afternoon, family just left & I wanted to be ready for tomorrow. I reviewed the assignment again & figured I’d pop open app.viam.com in preparation for starting the assignment in the morning. Since I’d already played with a rover, I was somewhat familiar with the UI of app.viam.com & generally aware that I could add devices/machines. I figured before I dive in, why not tool around for a hot second. I figured I’d spend a few minutes reacquainting myself with the app to get my juices flowing. 30 minutes later I’d accidentally had too much fun. I added my MacBook, setup my camera, added a model & vision service, trained the model on some samples of “me” and was about to start mucking around with the SDK when I caught myself & realized what I was doing & how far I’d gone. Kinda just outta 2nd nature I took some notes as I was going. I figured since I’d accidentally kinda “started” - I’d track this as my first session.

Setup Notes

  • “Add Machine” would make sense on the dashboard; or at least a link to it if you have few or no machines
  • The “Add Machine” button UX kinda sucks; you have to fill in the name & click “Add Machine” a 2nd time
  • Homebrew install was interesting; lots of dependencies; curious to dive deeper there
    • Placing default config at /opt/homebrew/etc/viam.json makes sense; kinda
    • Would be good to put some instructions somewhere that to run as a cloud-connected service - you’ll want to copy viam-[machine]-main.json to /opt/homebrew/etc/viam.json
    • There were quite a few messages related to LLVM & LLD - that warrant deeper looking at. Not sure if they’re relevant
  • Interesting that watchman is added as a service when you boot viam-server as a service
    • Makes me wonder if config or other changes may be firing hot reloads
  • Would be nice to be able to manually clear logs when doing config; to see if new errors pop up
  • I actually did the install on 2 machines in parallel-ish. One machine was freshly rebuilt, so I didn’t have oh-my-zsh setup, nor Homebrew installed yet. As brew install ran on one I was able to setup & the other & start the brew install; ~5m delta. Didn’t bother to connect 2nd machine in the app though.

Thoughts & Ideas

  • I’m gonna have to do actual training for a model & take action for some purpose
    • 1: Do the assignment first!!!! Don’t get carried away… stick to the assignment!!!
    • Stretch goals
      • Habit Hacker - trained models on observed behavior
        • Hydration Minder
        • Ah ah ah! Stop doing that!
      • Coop Keeper
      • Crop Keeper
      • Castle Keeper
  • I did the bare metal install for MacOS which was interesting
    • Brew managed native service; really bare metal machine; interesting for some cases & dev
  • Want to do Linux install in Docker/Podman containers
    • Pass devices to container; container based service lifecycle
    • Easier for testing multiple configurations? Branches?
    • Multiple servers running at the same time?
  • SUPER excited to get my RaspberryPi connected & start playing with that
  • I don’t want to stop at the assignment; I want to go deeper & start building out actual use-cases
    • Automate the entirety of chicken coop operations
    • Automate identification & reporting on defensible space & wildfire threats
    • Automate identification & reporting of citrus health & recommendations

Session 2 (the assignment): Tuesday April 8th, Start 1355 | End 1655

After getting derailed all morning by calls, friends in need & workers at the house, I’m finally ready to tackle the assignment. I feel like it’s appropriate to note as I feel the pressure to get after it as one might who’s already got a lot on their plate and their EM is asking them to check out Viam.

To give this a fair shot on timing, I’ve decided to wipe out the stuff I did yesterday before I start the clock - and start clean; actually referencing the tasks & creating things with intent.

  • Prerequisites
    • I’ve already watched the intro video from Elliot, perused the general docs & setup an account
    • I’ve skim watched the 42m video but I’ll now really watch it in-depth; I’ll start the clock on the assignment once I’ve completed the video.
      • video shows an some interesting evolution on the UI/UX
      • I like that robots is now machines - that would have been my suggestion
    • I see that task 3 includes the module registry so I’ll want to review the docs on modules; starting with the hello-world-module
      • Need to get setup with Viam CLI. Took ~10s.
      • Noted: our module is supposed to be a sensor.
  1. Task 1: Vision - 1m
    1. Unlike the video demo; camera paths don’t show up; followed the recommendation to add webcam-discovery
  2. Task 2: Cloud Integration - 1m
    1. Pretty simple and I can see files piling up; bumped to 1/sec
    2. Curious about cost & accrual of data??? How quickly can I blow this up???
    3. Easy to see how you add stuff to a training set & setup with a model; can’t wait to try it.
  3. Task 3: Person sensor - 2h 43m (with break, notes & cleanup time)
    1. Since we’re doing detection, I’ll setup a model now
    2. Also added vision service to interact with our model and give us a person with min confidence level
    3. Had to set a namespace for my org (or get the ID; opted for namespace)
    4. I did create a new VSCode workspace for my sensor; but saved the workspace outside the sensor directory cause I wanted to keep it pretty pristine of those kinds of artifacts.
    5. Looking at what the module generator gives us; we’ll need to add our logic to get_readings
    6. Looks like we need to make sure that we update validate_config and reconfigure to get dependencies
    7. Looks like we gotta find docs for the vision service & how to interact with it
      1. Took a sec to figure out where to look for docs specific to Vision API - but proper linking in PythonDoc helped
    8. — 1530 break time — 1545 resume
    9. Followed the guide for testing a local module; I added my new module & things started going sideways.
      1. Following the API instructions, why can’t it find my camera-1 that’s definitely what it’s referenced as
      2. Once I reference the camera via dependencies & grab the name from the object, why does that stop throwing errors but seem to blow up the whole pipeline?
      3. Restarting the service from Viam app takes a long time to do a clean shutdown. WTF? Several minutes here
      4. Restarted via Homebrew instead… Tired of waiting
      5. Ooooh interesting… Vision.get_resource_name gives us a VisionClient; not obvious; we’re gonna have to cast it if we can
        1. Looks like When you’re using ResourceName you might get the wrong class, so yes, you can cast: https://python.viam.dev/autoapi/viam/robot/client/index.html#viam.robot.client.RobotClient.get_component
      6. Took a moment to figure out that restarting my local module DOES pick up changes; I didn’t see that before rebooting my local server - but now it’s working
      7. Feature complete at 1645
      8. Cleanup started
      9. 1655 done
  4. Task 4: Describe your work
    1. Since I had so much fun playing with the platform, I thought it would be fun to present my work as I might if I were introducing the platform to a prospect EM who's concerned about what it takes to kick-the-tires; addressing not only the concerns of basic onboarding but also teasing ahead what else might be possible with minimal additional effort.
    2. Part of Task 2 was to consider, "How could we use this data to possibly create a custom model?". I thought it would also be fun to present this as I would to a prospect & take them on live demo, building a training set from captured images, training a model & adding it to our config - perhaps able to identify whether or not a person is drinking? I want to present in a way that excites the audience & get the creative juices flowing.
    3. Note to self; try to stay a bit more formal than you'd typically do for an eng demo/pres. Familiarity breeds contempt.

Assignment Review

Overall, this was a very fun & rewarding exercise & left me hungry for more.

I was surprised that it took me only 3h including my 15m break to go from nothing to a completed solution. I would say the bulk of that time went to looking up API docs to figure out exactly what I needed to interact with my vision service. I also blasted a bunch of time when I was getting an instance of VisionClient and even after resolving that - I wasn’t seeing changes in the app. Not even changed error logs. So I figured something was borked & restarted the local viam-server. That made an instant difference and I was able to get back on track. Overall, that’s not really surprising and certainly didn’t turn me off in any way.

I wanted to take a clean-room approach to this, so I took the perspective of an overwhelmed, or perhaps jr. dev & resisted the urge to follow my instinctual path of finding & referencing prior art. I assumed myself to be someone who either doesn’t know to do that, doesn’t know how - or is too tired to hunt things down. I used VSCode for working on my sensor & autocompletion based on imported dependencies saved a lot of time & effort. I expect any decent IDE to offer that for whatever your language of choice is.

In terms of module development, I thought that was pretty slick from a DevEx standpoint. It was SUPER straightforward & it was very clear to me where to do the implementation. The config setup took me a bit; mostly just figuring out how that worked and how to get the class I was expecting to be using - but once I recognized how that all worked, boom. I found the SDK code samples to be an immediately helpful resource. Once I had generated the module stubs & I could compare that with the SDK sample it was obvious what classes I needed & I could pretty much "see" the implementation in my head.

I would be super stoked to try all the other language SDKs too. Perhaps, as I continue building against my stretch goals, I'll try them out. I love JavaScript/TypeScript so I'll probably start there.

As I was doing some cleanup & nearing completion I did use the model registry & GitHub to find prior-art for a person detector sensor and was pretty surprised to see how similar they were to what I came up with. I didn’t bother with elaborate error handling, but it was kinda creepy to see how close our approaches & code were. The fact that I was able to find prior art in the registry and on GitHub is really strong selling point IMHO. If you’re just diving in & trying to validate what you’re doing - or get inspiration, there’s a BUNCH of modules to check out - which really surprised me for something this early-stage.

I'm leaving this exercise totally pumped - on both personal & professional levels. Just like MongoDB, I am absoluteley using Viam for personal projects & will be recommending to any matching professional contexts or challenges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment