Skip to content

Instantly share code, notes, and snippets.

@kevinrobinson
Last active January 9, 2021 16:09

Revisions

  1. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 3 additions and 1 deletion.
    4 changes: 3 additions & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -93,6 +93,8 @@ So my closing three thoughts are:

    Of course, I'm writing you because I admire your work tremendously, and would really value any feedback or thoughts you had the time to share. Keep doing (and sharing) your awesome work! :)

    ```
    -Kevin
    [email protected]
    @krob
    @krob
    ```
  2. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -85,11 +85,11 @@ This code is spectacularly well-crafted and reading it has been a fantastic lear

    So my closing three thoughts are:

    - What if we reified effects, like in-flight server requests, into state? And then the function that took prev-state -> new-state could consider the state of effects as part of producing the new state value, and it could also perform effects as needed (e.g., abort in-flight requests that aren't relevant anymore, make requests for additional data, kick off another iteration of a simulation).
    - What if we reified effects, like in-flight server requests, into state? This would let the function that took `(prev-state) -> new-state` consider the state of effects as part of producing the new state value, and it could also perform effects as needed (e.g., abort or make server requests, kick off another iteration of a simulation).

    - Would state transitions (including state of effects) be more natural if they fit within the lifecycle of view components?

    - The global state atom is being mutated in-place. That works well for the representation of the UI state, since the DOM ultimately is a place that must be mutated. But it doesn't work as well for a sophisticated client-side view of server state. What if we split out the full client-side view of server state (including optimistic updates) to an immutable data store in the browser?
    - The global state atom is being mutated in-place, which works well for a representation of the UI state since the DOM ultimately is a place that must be mutated. But it doesn't work as well for a sophisticated client-side view of server state. What if we split out the full client-side view of server state (including facts like optimistic updates) to an immutable data store in the browser?

    Of course, I'm writing you because I admire your work tremendously, and would really value any feedback or thoughts you had the time to share. Keep doing (and sharing) your awesome work! :)

  3. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 3 additions and 3 deletions.
    6 changes: 3 additions & 3 deletions circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -85,11 +85,11 @@ This code is spectacularly well-crafted and reading it has been a fantastic lear

    So my closing three thoughts are:

    A. What if we reified effects, like in-flight server requests, into state? And then the function that took prev-state -> new-state could consider the state of effects as part of producing the new state value, and it could also perform effects as needed (e.g., abort in-flight requests that aren't relevant anymore, make requests for additional data, kick off another iteration of a simulation).
    - What if we reified effects, like in-flight server requests, into state? And then the function that took prev-state -> new-state could consider the state of effects as part of producing the new state value, and it could also perform effects as needed (e.g., abort in-flight requests that aren't relevant anymore, make requests for additional data, kick off another iteration of a simulation).

    B. Would state transitions (including state of effects) be more natural if they fit within the lifecycle of view components?
    - Would state transitions (including state of effects) be more natural if they fit within the lifecycle of view components?

    C. The global state atom is being mutated in-place. That works well for the representation of the UI state, since the DOM ultimately is a place that must be mutated. But it doesn't work as well for a sophisticated client-side view of server state. What if we split out the full client-side view of server state (including optimistic updates) to an immutable data store in the browser?
    - The global state atom is being mutated in-place. That works well for the representation of the UI state, since the DOM ultimately is a place that must be mutated. But it doesn't work as well for a sophisticated client-side view of server state. What if we split out the full client-side view of server state (including optimistic updates) to an immutable data store in the browser?

    Of course, I'm writing you because I admire your work tremendously, and would really value any feedback or thoughts you had the time to share. Keep doing (and sharing) your awesome work! :)

  4. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -57,7 +57,7 @@ Discussion
    ---
    It'll be great to get some more eyes on this and verify my understanding, but I'm going to reflect a bit, optimistically assuming that it's at least roughly correct. :)

    As framing, my instinct is that what's really important when evaluating front-end architectures is 1) the flexibilty and speed when adding new product features, and 2) encouraging specific or local abstractions, and avoiding abstractions that might not fit what we need for any given product feature. I think aggressively decoupling and abstracting at the local level is important, but I'm cautious about defining any seams that enforce this at the global level, since there's a much greater chance that the assumptions of that abstraction will be wrong. Refactoring is easy and mechanical; re-abstracting is difficult and requires deep understanding.
    As framing, my instinct is that what's most important when evaluating front-end architectures is that it maintains the flexibilty and speed to add unanticipated product features. My bias for how this is accomplished is that I think aggressively decoupling and abstracting at the local level is important, but I'm cautious about defining seams that enforce abstractions at the global level, since there's a much greater chance that the assumptions of those abstractions will be wrong. Refactoring is easy and mechanical; re-abstracting is difficult and requires deep understanding.

    1. This project is amazing, and has a lot of really creative and powerful ideas to learn from! The pervasive use of pure functions leads to being able to reason about the program with only local knowlege in most of the app. Almost all rendering code is isolated from the code that changes app state (thanks React!), which has the enormous benefit of letting us mostly ignore time in rendering code. The functional idea of "data structures first" is really clear here, with pure static functions parsing, interpreting and decorating them. The organization of the model module and functions is intuitive and natural, with any behaviors or processes on that data defined separately. The edges of the browser like mouse input and URL state changes are pushed to the edges, and there are clear seams there. Using pub-sub points as the way for view code to send events into the rest of the app is really clear, and the mechanism of `handler(e)` -> `"update state in place" and "perform other effects like making requests"` makes it straightforward to reason about these state changes in isolation.

  5. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 3 additions and 4 deletions.
    7 changes: 3 additions & 4 deletions circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -55,10 +55,11 @@ Ok, so that's about it for that code path. One more note before popping up the

    Discussion
    ---
    It'll be great to get some more eyes on this and verify my understanding, but I'm going to reflect a bit, optimistically assuming that it's at least roughly correct. :)

    So, what are the big picture takeways here? It'll be great to get some more eyes on this and verify my understanding, but I'm going to reflect a bit, optimistically assuming that it's at least roughly correct. :)
    As framing, my instinct is that what's really important when evaluating front-end architectures is 1) the flexibilty and speed when adding new product features, and 2) encouraging specific or local abstractions, and avoiding abstractions that might not fit what we need for any given product feature. I think aggressively decoupling and abstracting at the local level is important, but I'm cautious about defining any seams that enforce this at the global level, since there's a much greater chance that the assumptions of that abstraction will be wrong. Refactoring is easy and mechanical; re-abstracting is difficult and requires deep understanding.

    1. This project is amazing, and has a lot of really creative and powerful ideas to learn from! The pervasive use of pure functions leads to being able to reason about the program with only local knowlege in most of the app. Almost all rendering code is isolated from the code that changes app state (thanks React!), which has the enormous benefit of letting us mostly ignore time in rendering code. The functional idea of "data structures first" is really clear here, with pure static functions parsing, interpreting and decorating them. The organization of the model module and functions is intuitive and natural, with any behaviors or processes on that data defined separately. Using pub-sub points as the way for view code to send events into the rest of the app is really clear, and the mechanism of `handler(e)` -> `"update state in place" and "perform other effects like making requests"` makes it straightforward to reason about these state changes in isolation.
    1. This project is amazing, and has a lot of really creative and powerful ideas to learn from! The pervasive use of pure functions leads to being able to reason about the program with only local knowlege in most of the app. Almost all rendering code is isolated from the code that changes app state (thanks React!), which has the enormous benefit of letting us mostly ignore time in rendering code. The functional idea of "data structures first" is really clear here, with pure static functions parsing, interpreting and decorating them. The organization of the model module and functions is intuitive and natural, with any behaviors or processes on that data defined separately. The edges of the browser like mouse input and URL state changes are pushed to the edges, and there are clear seams there. Using pub-sub points as the way for view code to send events into the rest of the app is really clear, and the mechanism of `handler(e)` -> `"update state in place" and "perform other effects like making requests"` makes it straightforward to reason about these state changes in isolation.

    2. It seems like as the state atom grows, the possible state transitions grow more complex as well. It feels to me like even in a project of this size, the enormous benefits of using plain data structures like maps and seqs start to become outweighed by the unbounded transitions that can be performed on them. My first instinct would be to abstract this by creating helper functions that transition parts of the state atom, and then composing those. Or by making a few number of higher-level functions for semantic transitions like `to-logged-out` or `to-project-page`. This codebase has a fair bit of this already, which is very clear and simple to understand. The comment about needing help to simplify this rings true to me, but I'm more concerned about whether the semantics of the state transition are valid (including that they're composed correctly), than using middleware mechanics to change how they are composed. I wonder also if by using the global state atom, we actually make this problem much worse. It'd be interesting to look in more depth another time at how these state transitions work for this specific app, and whether there are natural seams to break this up along (even if they are stored in a single global atom). These seams might be along either the entities that are present (e.g., "transition between there being a current project or not"), or they might be along the pages themselves (e.g., "transition to state for dominant-component X").

    @@ -71,8 +72,6 @@ So, what are the big picture takeways here? It'll be great to get some more eye

    Future thoughts
    ---
    My instinct is that the most important thing in evaluating front-end architectures is keeping the perspective on what's really important: flexibilty and speed with adding new product features, and limiting coupling between distint product features as a way to avoid premature abstraction. But reducing coupling isn't an abstract goal, it's in service of being able to make new features more quickly, or change existing code to meet requirements that were unknown at the time. Refactoring is easy and mechanical; re-abstracting is difficult and requires deep understanding.

    The problems that affect me and my colleagues the most day-to-day are the "dynamic process" parts of the UI, and true distributed systems problems. I don't think applying ideas in CSP about how to model dynamic processes is a great fit for UIs, since writing correct concurrent systems is a different kind of work, and in my experience, not the kind of work that usually has the best product payoff for engineers that live close to the product and user experience.

    I have a similar feeling about true distributed systems problems of data consistency. In practice, most products I've worked on benefitted by being explicit in their avoidance of these problems, since the engineering investment wasn't worth it for the product payoff. Simpler approximations are almost always sufficient. Like taking a snapshot of the server state and holding it as immutable for a period of seconds or minutes, predicting that the likelihook of it changing underneath the user is very low and the path to recovery when it does happen is very straightforward. Or embracing eventual consistency and decoupling, and allowing data to be updated in different views within seconds of each other rather than synchronously. Or by investing in improvements to the API serving layer so that server response times are reliably within 100s of milliseconds and it's more acceptable to always get fresh data or wait for ack responses.
  6. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 3 additions and 3 deletions.
    6 changes: 3 additions & 3 deletions circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -86,11 +86,11 @@ This code is spectacularly well-crafted and reading it has been a fantastic lear

    So my closing three thoughts are:

    a. What if we reified effects, like in-flight server requests, into state? And then the function that took prev-state -> new-state could consider the state of effects as part of producing the new state value, and it could also perform effects as needed (e.g., abort in-flight requests that aren't relevant anymore, make requests for additional data, kick off another iteration of a simulation).
    A. What if we reified effects, like in-flight server requests, into state? And then the function that took prev-state -> new-state could consider the state of effects as part of producing the new state value, and it could also perform effects as needed (e.g., abort in-flight requests that aren't relevant anymore, make requests for additional data, kick off another iteration of a simulation).

    b. Would state transitions (including state of effects) be more natural if they fit within the lifecycle of view components?
    B. Would state transitions (including state of effects) be more natural if they fit within the lifecycle of view components?

    c. The global state atom is being mutated in-place. That works well for the representation of the UI state, since the DOM ultimately is a place that must be mutated. But it doesn't work as well for a sophisticated client-side view of server state. What if we split out the full client-side view of server state (including optimistic updates) to an immutable data store in the browser?
    C. The global state atom is being mutated in-place. That works well for the representation of the UI state, since the DOM ultimately is a place that must be mutated. But it doesn't work as well for a sophisticated client-side view of server state. What if we split out the full client-side view of server state (including optimistic updates) to an immutable data store in the browser?

    Of course, I'm writing you because I admire your work tremendously, and would really value any feedback or thoughts you had the time to share. Keep doing (and sharing) your awesome work! :)

  7. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 19 additions and 9 deletions.
    28 changes: 19 additions & 9 deletions circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -68,22 +68,32 @@ So, what are the big picture takeways here? It'll be great to get some more eye

    5. Using channels for tracking server communication feel like a mismatch for the problem here. This isn't about their local use, but about emitting events about state changes for requests on a pub-sub point. Using the example of the `navigation` controller's making calls for the `dashboard`, it creates a parallel kind of application state that isn't made concrete in the state atom. The application state can't actually be entirely described by the state atom, there's also hidden state with in-flight server requests, and even further, goroutines that controller code has queued up, waiting to perform additional requests and state transitions when the server requests do complete. And there's no way to abort in-flight requests or scheduled effects based other state changes that occur. As an example, after the `:dashboard` navigation event is emitted, let's say the user navigates somewhere else. The first dashboard request is still in flight, and when it completes, there is code already queued up to respond to that and make additional requests. These requests are irrelevant now (the user has told us they don't care about that data anymore), but there's no way to abort those queued effects. You can see this complexity leaking into the API controller code with those guards. Note: I'm not arguing against channels in general, in fact I think the mechanics or Promises/callbacks/channels are not particularly important. In the way channels are used locally, like with `:get builds` in the navigation controller, it's not any different than using Promises or callbacks, that's cool. If we needed some more complex composition, local use of channels and goroutines seems awesome (although I haven't yet seen a compelling real-world use case for channels over Promises, other than the Golang example of fanning out and taking the fastest responders, and that doesn't translate well to the browser). My point here is about the hidden state of in-flight server requests, and code that's queued up to run, neither of which are encoded in the state atom.

    Takeaway

    Future thoughts
    ---
    This code is spectacularly well-crafted and reading it has been a fantastic learning experience. I imagine I'll keep following the work you are doing, as in my mind you're doing some of the leading creative work in this area, and the kind of work that has the potential to really push us forward and help the whole field.
    My instinct is that the most important thing in evaluating front-end architectures is keeping the perspective on what's really important: flexibilty and speed with adding new product features, and limiting coupling between distint product features as a way to avoid premature abstraction. But reducing coupling isn't an abstract goal, it's in service of being able to make new features more quickly, or change existing code to meet requirements that were unknown at the time. Refactoring is easy and mechanical; re-abstracting is difficult and requires deep understanding.

    The problems that affect me and my colleagues the most day-to-day are the "dynamic process" parts of the UI, and true distributed systems problems. I don't think applying ideas in CSP about how to model dynamic processes is a great fit for UIs, since writing correct concurrent systems is a different kind of work, and in my experience, not the kind of work that usually has the best product payoff for engineers that live close to the product and user experience.

    I have a similar feeling about true distributed systems problems of data consistency. In practice, most products I've worked on benefitted by being explicit in their avoidance of these problems, since the engineering investment wasn't worth it for the product payoff. Simpler approximations are almost always sufficient. Like taking a snapshot of the server state and holding it as immutable for a period of seconds or minutes, predicting that the likelihook of it changing underneath the user is very low and the path to recovery when it does happen is very straightforward. Or embracing eventual consistency and decoupling, and allowing data to be updated in different views within seconds of each other rather than synchronously. Or by investing in improvements to the API serving layer so that server response times are reliably within 100s of milliseconds and it's more acceptable to always get fresh data or wait for ack responses.

    I think the process and dynamic parts of UI code is where the hard parts are that plague us most day-to-day. And taking the strengths of CSP to model dynamic processes, and applying them to UIs isn't a great fit, since writing correct concurrent systems is a different kind of work, and in my experience, not the kind of work that usually has the best product payoff for engineers that live close to the product and user experience.
    But if we do want to make a product or feature that functions as a true distributed system, that of all things should be split out from the rest of the application. :) The idea that feels the most promising is making the "view of server state" immutable, and, within the application, recording facts about server state and changes in client state. This way view code can compute views of those facts, like merging optimistic client updates with the view of server state. And there's a first-class way to deal with failure, like retracting facts about optimistic updates that the server later rejects, etc. There's some interesting related work to get a Datomic-style in-memory database in the browser [http://tonsky.me/blog/decomposing-web-app-development/], which is what I picture the database looking like. But the source in that chat example doesn't use DataScript to store state about effects, and in fact the state about server calls for `load-user` is kept on the stack in a separate goroutine, unknown to functions trying to transact database state. I think the real architectural achievement is when this kind of design is simple enough to work with that these guidelines still seem natural, intuitive, and unburdensome for small forms or chat apps.

    The other parts of UI programming that I think are truly difficult are the true distributed systems problems of data consistency. But in practice, most applications don't actually need to solve this, or the engineering investment isn't worth it for the product payoff. Simpler approximations are almost always sufficient. Like taking a snapshot of the server state and holding it as immutable for a period of seconds or minutes, predicting that the likelihook of it changing underneath the user is very low and the path to recovery when it does happen is very straightforward. Or embracing eventual consistency and decoupling, and allowing data to be updated in different views within seconds of each other rather than synchronously. Or by investing in improvements to the API serving layer so that server response times are reliably within 100s of milliseconds and more extensive browser/device-side caching is not worth the complexity.

    But if we do want to make a product or feature that functions as a true distributed system, that of all things should be split out from the rest of the application. :) The idea that feels the most promising is making the "view of server state" immutable, and, within the application, recording facts about server state and changes in client state. This way view code can compute views of those facts, like merging optimistic client updates with the view of server state. And there's a first-class way to deal with failure, like retracting facts about optimistic updates that the server later rejects, etc. I think the real architectural achievement is when this design is simple enough to work with that these guidelines still seem natural, intuitive, and unburdensome for small forms or chat apps.
    Takeaway
    ---
    This code is spectacularly well-crafted and reading it has been a fantastic learning experience. I'm sure I'll keep following the work you are doing, as in my mind you're doing some of the leading creative work in this area, and the kind of work that has the potential to really push us forward and help the whole field.

    So my closing two thoughts are:
    So my closing three thoughts are:

    1. What if we reified effects, like in-flight server requests, into state? And then the function that took prev-state -> new-state could consider the state of effects as part of producing the new state value, and it could also perform effects as needed (e.g., abort in-flight requests that aren't relevant anymore, make requests for additional data, kick off another iteration of a simulation).
    a. What if we reified effects, like in-flight server requests, into state? And then the function that took prev-state -> new-state could consider the state of effects as part of producing the new state value, and it could also perform effects as needed (e.g., abort in-flight requests that aren't relevant anymore, make requests for additional data, kick off another iteration of a simulation).

    2. The global state atom is being mutated in-place. That works well for the representation of the UI state, since the DOM ultimately is a place that must be mutated. But it doesn't work as well for a sophisticated client-side view of server state. What if we split out the full client-side view of server state (including optimistic updates) to an immutable data store in the browser?
    b. Would state transitions (including state of effects) be more natural if they fit within the lifecycle of view components?

    c. The global state atom is being mutated in-place. That works well for the representation of the UI state, since the DOM ultimately is a place that must be mutated. But it doesn't work as well for a sophisticated client-side view of server state. What if we split out the full client-side view of server state (including optimistic updates) to an immutable data store in the browser?

    Of course, I'm writing you because I admire your work tremendously, and would really value any feedback or thoughts you had the time to share. Keep doing (and sharing) your awesome work! :)

    -Kevin
    -Kevin
    [email protected]
    @krob
  8. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 0 additions and 2 deletions.
    2 changes: 0 additions & 2 deletions circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -72,8 +72,6 @@ Takeaway
    ---
    This code is spectacularly well-crafted and reading it has been a fantastic learning experience. I imagine I'll keep following the work you are doing, as in my mind you're doing some of the leading creative work in this area, and the kind of work that has the potential to really push us forward and help the whole field.

    My instinct is that all front-end architectures are missing the perspective of orienting and evaluating themselves on what's really important: flexibilty and speed with adding new product features, and limiting coupling between distint product features as a way to avoid premature abstraction. But reducing coupling isn't an abstract goal, it's in service of being able to make new features more quickly, or change existing code to meet requirements that were unknown at the time. Refactoring is easy and mechanical; re-abstracting is difficult and requires deep understanding.

    I think the process and dynamic parts of UI code is where the hard parts are that plague us most day-to-day. And taking the strengths of CSP to model dynamic processes, and applying them to UIs isn't a great fit, since writing correct concurrent systems is a different kind of work, and in my experience, not the kind of work that usually has the best product payoff for engineers that live close to the product and user experience.

    The other parts of UI programming that I think are truly difficult are the true distributed systems problems of data consistency. But in practice, most applications don't actually need to solve this, or the engineering investment isn't worth it for the product payoff. Simpler approximations are almost always sufficient. Like taking a snapshot of the server state and holding it as immutable for a period of seconds or minutes, predicting that the likelihook of it changing underneath the user is very low and the path to recovery when it does happen is very straightforward. Or embracing eventual consistency and decoupling, and allowing data to be updated in different views within seconds of each other rather than synchronously. Or by investing in improvements to the API serving layer so that server response times are reliably within 100s of milliseconds and more extensive browser/device-side caching is not worth the complexity.
  9. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -66,7 +66,7 @@ So, what are the big picture takeways here? It'll be great to get some more eye

    4. Continuing further, moving the orchestration of API requests into a component gives it a natural lifecycle for avoiding race conditions, even with a global state atom. For example, when the `dashboard` component is unmounted, any of its requests could be aborted. The callback code for updating the state atom can be owned and contained within the component's lifecycle. This provides the kind of guard that's accomplished now by manual checks in the API controller, and in a more natural way. As the components grow more complex, the mechanics for this are the bits of code that are split out to keep the complexity low. But by using a design that makes it hard to see the "why" and the impact on the product and UX, we actually make the problem of making new products and features harder. And the "why" and product can't be encoded or understood without thinking about view components. My instinct is that we shouldn't try to split the "why" out of view components. It's only the "how" details of server communication, mechanics of cache lookups (but not expiration and invalidation logic), logic of parsing model data, etc. should be split out.

    5. Using channels for tracking server communication feel like a mismatch for the problem here. This isn't about their local use, but about putting events about emitting events about state changes for requests on a pub-sub point. Using the example of the `navigation` controller's making calls for the `dashboard`, it creates a parallel kind of application state that isn't made concrete in the state atom. The application state can't actually be entirely described by the state atom, there's also hidden state with in-flight server requests, and even further, goroutines that controller code has queued up, waiting to perform additional requests and state transitions when the server requests do complete. And there's no way to abort in-flight requests or scheduled effects based other state changes that occur. As an example, after the `:dashboard` navigation event is emitted, let's say the user navigates somewhere else. The first dashboard request is still in flight, and when it completes, there is code already queued up to respond to that and make additional requests. These requests are irrelevant now (the user has told us they don't care about that data anymore), but there's no way to abort those queued effects. You can see this complexity leaking into the API controller code with those guards. Note: I'm not arguing against channels in general, in fact I think the mechanics or Promises/callbacks/channels are not particularly important. In the way channels are used locally, like with `:get builds` in the navigation controller, it's not any different than using Promises or callbacks, that's cool. If we needed some more complex composition, local use of channels and goroutines seems awesome (although I haven't yet seen a compelling real-world use case for channels over Promises, other than the Golang example of fanning out and taking the fastest responders, and that doesn't translate well to the browser). My point here is about the hidden state of in-flight server requests, and code that's queued up to run, neither of which are encoded in the state atom.
    5. Using channels for tracking server communication feel like a mismatch for the problem here. This isn't about their local use, but about emitting events about state changes for requests on a pub-sub point. Using the example of the `navigation` controller's making calls for the `dashboard`, it creates a parallel kind of application state that isn't made concrete in the state atom. The application state can't actually be entirely described by the state atom, there's also hidden state with in-flight server requests, and even further, goroutines that controller code has queued up, waiting to perform additional requests and state transitions when the server requests do complete. And there's no way to abort in-flight requests or scheduled effects based other state changes that occur. As an example, after the `:dashboard` navigation event is emitted, let's say the user navigates somewhere else. The first dashboard request is still in flight, and when it completes, there is code already queued up to respond to that and make additional requests. These requests are irrelevant now (the user has told us they don't care about that data anymore), but there's no way to abort those queued effects. You can see this complexity leaking into the API controller code with those guards. Note: I'm not arguing against channels in general, in fact I think the mechanics or Promises/callbacks/channels are not particularly important. In the way channels are used locally, like with `:get builds` in the navigation controller, it's not any different than using Promises or callbacks, that's cool. If we needed some more complex composition, local use of channels and goroutines seems awesome (although I haven't yet seen a compelling real-world use case for channels over Promises, other than the Golang example of fanning out and taking the fastest responders, and that doesn't translate well to the browser). My point here is about the hidden state of in-flight server requests, and code that's queued up to run, neither of which are encoded in the state atom.

    Takeaway
    ---
  10. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -64,7 +64,7 @@ So, what are the big picture takeways here? It'll be great to get some more eye

    3. Similarly, the architecture lines for the channels and the controllers (e.g., navigation/API/controls/errors) feel a bit forced, like they are organizing parts of the app based on the "what" or "how" rather than the "why". As an example, the code for responding to most API calls is organized into a single `api controller` module. The context of the "why" is hard to see - why the return value is what it is, why the code is doing this, or where it fits in the context of the app and user experience. The best place to see the "why" is in the navigation controller for the `:dashboard`. But it's in a `navigation` module, mixed with the "why" of the other pages. It seems more natural to me that the holistic higher-level "why" should be fully described as part of the `dashboard` component. That component might make use of common `api` or `navigation` code bits to perform its work, or split out the mechanics and "how" into separate modules. But it feels to me like the important semantic parts, the meaning and flow of the product, should be expressed directly and wholly in one place. My instinct is that the "dashboard" component is actually the right place for this.

    4. Continuing further, moving the orchestration of API requests into a component gives it a natural lifecycle for avoiding race conditions, even with a global state atom. For example, when the `dashboard` component is unmounted, any of its requests could be aborted. The callback code for updating the state atom can be owned and contained within the component's lifecycle. This provides the kind of guard that's accomplished now by manual checks in the API controller, and in a more natural way. As the components grow more complex, the mechanics for this are the bits of code that are split out to keep the complexity low. But using an architecture that makes it hard to see the "why" and the impact on the product and UX (which can only be encoded in view components), we actually make the problem of making new products and features harder. My strong feeling (the one that prompted me to study your work for ideas), is that these things shouldn't be decoupled; only the "how" details of server communication, mechanics of caching, logic of parsing model data, etc. should be split out.
    4. Continuing further, moving the orchestration of API requests into a component gives it a natural lifecycle for avoiding race conditions, even with a global state atom. For example, when the `dashboard` component is unmounted, any of its requests could be aborted. The callback code for updating the state atom can be owned and contained within the component's lifecycle. This provides the kind of guard that's accomplished now by manual checks in the API controller, and in a more natural way. As the components grow more complex, the mechanics for this are the bits of code that are split out to keep the complexity low. But by using a design that makes it hard to see the "why" and the impact on the product and UX, we actually make the problem of making new products and features harder. And the "why" and product can't be encoded or understood without thinking about view components. My instinct is that we shouldn't try to split the "why" out of view components. It's only the "how" details of server communication, mechanics of cache lookups (but not expiration and invalidation logic), logic of parsing model data, etc. should be split out.

    5. Using channels for tracking server communication feel like a mismatch for the problem here. This isn't about their local use, but about putting events about emitting events about state changes for requests on a pub-sub point. Using the example of the `navigation` controller's making calls for the `dashboard`, it creates a parallel kind of application state that isn't made concrete in the state atom. The application state can't actually be entirely described by the state atom, there's also hidden state with in-flight server requests, and even further, goroutines that controller code has queued up, waiting to perform additional requests and state transitions when the server requests do complete. And there's no way to abort in-flight requests or scheduled effects based other state changes that occur. As an example, after the `:dashboard` navigation event is emitted, let's say the user navigates somewhere else. The first dashboard request is still in flight, and when it completes, there is code already queued up to respond to that and make additional requests. These requests are irrelevant now (the user has told us they don't care about that data anymore), but there's no way to abort those queued effects. You can see this complexity leaking into the API controller code with those guards. Note: I'm not arguing against channels in general, in fact I think the mechanics or Promises/callbacks/channels are not particularly important. In the way channels are used locally, like with `:get builds` in the navigation controller, it's not any different than using Promises or callbacks, that's cool. If we needed some more complex composition, local use of channels and goroutines seems awesome (although I haven't yet seen a compelling real-world use case for channels over Promises, other than the Golang example of fanning out and taking the fastest responders, and that doesn't translate well to the browser). My point here is about the hidden state of in-flight server requests, and code that's queued up to run, neither of which are encoded in the state atom.

  11. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -64,7 +64,7 @@ So, what are the big picture takeways here? It'll be great to get some more eye

    3. Similarly, the architecture lines for the channels and the controllers (e.g., navigation/API/controls/errors) feel a bit forced, like they are organizing parts of the app based on the "what" or "how" rather than the "why". As an example, the code for responding to most API calls is organized into a single `api controller` module. The context of the "why" is hard to see - why the return value is what it is, why the code is doing this, or where it fits in the context of the app and user experience. The best place to see the "why" is in the navigation controller for the `:dashboard`. But it's in a `navigation` module, mixed with the "why" of the other pages. It seems more natural to me that the holistic higher-level "why" should be fully described as part of the `dashboard` component. That component might make use of common `api` or `navigation` code bits to perform its work, or split out the mechanics and "how" into separate modules. But it feels to me like the important semantic parts, the meaning and flow of the product, should be expressed directly and wholly in one place. My instinct is that the "dashboard" component is actually the right place for this.

    4. Continuing further, moving the orchestration of API requests into a component gives it a natural lifecycle for avoiding race conditions, even with a global state atom. For example, when the `dashboard` component is unmounted, any of its requests could be aborted. The callback code for updating the state atom can be owned and contained within the component's lifecycle. This provides the kind of guard that's accomplished now by manual checks in the API controller, and in a more natural way. As the components grow more complex, the mechanics for this are the bits of code that are split out to keep the complexity low. But by divorcing the architecture from the product and UX (which can only be encoded in View components), we actually make the problem of making new products and features harder. These things shouldn't be decoupled, only the "how" details of server communication, mechanics of caching, logic of parsing model data, etc. should be split out.
    4. Continuing further, moving the orchestration of API requests into a component gives it a natural lifecycle for avoiding race conditions, even with a global state atom. For example, when the `dashboard` component is unmounted, any of its requests could be aborted. The callback code for updating the state atom can be owned and contained within the component's lifecycle. This provides the kind of guard that's accomplished now by manual checks in the API controller, and in a more natural way. As the components grow more complex, the mechanics for this are the bits of code that are split out to keep the complexity low. But using an architecture that makes it hard to see the "why" and the impact on the product and UX (which can only be encoded in view components), we actually make the problem of making new products and features harder. My strong feeling (the one that prompted me to study your work for ideas), is that these things shouldn't be decoupled; only the "how" details of server communication, mechanics of caching, logic of parsing model data, etc. should be split out.

    5. Using channels for tracking server communication feel like a mismatch for the problem here. This isn't about their local use, but about putting events about emitting events about state changes for requests on a pub-sub point. Using the example of the `navigation` controller's making calls for the `dashboard`, it creates a parallel kind of application state that isn't made concrete in the state atom. The application state can't actually be entirely described by the state atom, there's also hidden state with in-flight server requests, and even further, goroutines that controller code has queued up, waiting to perform additional requests and state transitions when the server requests do complete. And there's no way to abort in-flight requests or scheduled effects based other state changes that occur. As an example, after the `:dashboard` navigation event is emitted, let's say the user navigates somewhere else. The first dashboard request is still in flight, and when it completes, there is code already queued up to respond to that and make additional requests. These requests are irrelevant now (the user has told us they don't care about that data anymore), but there's no way to abort those queued effects. You can see this complexity leaking into the API controller code with those guards. Note: I'm not arguing against channels in general, in fact I think the mechanics or Promises/callbacks/channels are not particularly important. In the way channels are used locally, like with `:get builds` in the navigation controller, it's not any different than using Promises or callbacks, that's cool. If we needed some more complex composition, local use of channels and goroutines seems awesome (although I haven't yet seen a compelling real-world use case for channels over Promises, other than the Golang example of fanning out and taking the fastest responders, and that doesn't translate well to the browser). My point here is about the hidden state of in-flight server requests, and code that's queued up to run, neither of which are encoded in the state atom.

  12. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -62,7 +62,7 @@ So, what are the big picture takeways here? It'll be great to get some more eye

    2. It seems like as the state atom grows, the possible state transitions grow more complex as well. It feels to me like even in a project of this size, the enormous benefits of using plain data structures like maps and seqs start to become outweighed by the unbounded transitions that can be performed on them. My first instinct would be to abstract this by creating helper functions that transition parts of the state atom, and then composing those. Or by making a few number of higher-level functions for semantic transitions like `to-logged-out` or `to-project-page`. This codebase has a fair bit of this already, which is very clear and simple to understand. The comment about needing help to simplify this rings true to me, but I'm more concerned about whether the semantics of the state transition are valid (including that they're composed correctly), than using middleware mechanics to change how they are composed. I wonder also if by using the global state atom, we actually make this problem much worse. It'd be interesting to look in more depth another time at how these state transitions work for this specific app, and whether there are natural seams to break this up along (even if they are stored in a single global atom). These seams might be along either the entities that are present (e.g., "transition between there being a current project or not"), or they might be along the pages themselves (e.g., "transition to state for dominant-component X").

    3. Similarly, the architecture lines for the channels and the controllers (e.g., navigation/API/controls/errors) feel a bit forced, like they are organizing parts of the app based on the "what" or "how" rather than the "why". As an example, the code for responding to most API calls is organized into a single `api` module. The context of the "why" of the return value, or what this code is doing in the context of the app or the user experience is lost. The best place to see the "why" is in the navigation controller for the `:dashboard`. But it's in a `navigation` module, mixed with the "why" of the other pages. It seems much more natural to me that the holistic higher-level "why" should be fully described in the `dashboard` component. That component might make use of common `api` or `navigation` code bits to perform its work, or split out the mechanics and "how" into separate modules. But it feels to me like the important semantic parts, the meaning and flow of the product, should be expressed directly and wholly in one place. My instinct is that the "dashboard" component is actually the right place for this.
    3. Similarly, the architecture lines for the channels and the controllers (e.g., navigation/API/controls/errors) feel a bit forced, like they are organizing parts of the app based on the "what" or "how" rather than the "why". As an example, the code for responding to most API calls is organized into a single `api controller` module. The context of the "why" is hard to see - why the return value is what it is, why the code is doing this, or where it fits in the context of the app and user experience. The best place to see the "why" is in the navigation controller for the `:dashboard`. But it's in a `navigation` module, mixed with the "why" of the other pages. It seems more natural to me that the holistic higher-level "why" should be fully described as part of the `dashboard` component. That component might make use of common `api` or `navigation` code bits to perform its work, or split out the mechanics and "how" into separate modules. But it feels to me like the important semantic parts, the meaning and flow of the product, should be expressed directly and wholly in one place. My instinct is that the "dashboard" component is actually the right place for this.

    4. Continuing further, moving the orchestration of API requests into a component gives it a natural lifecycle for avoiding race conditions, even with a global state atom. For example, when the `dashboard` component is unmounted, any of its requests could be aborted. The callback code for updating the state atom can be owned and contained within the component's lifecycle. This provides the kind of guard that's accomplished now by manual checks in the API controller, and in a more natural way. As the components grow more complex, the mechanics for this are the bits of code that are split out to keep the complexity low. But by divorcing the architecture from the product and UX (which can only be encoded in View components), we actually make the problem of making new products and features harder. These things shouldn't be decoupled, only the "how" details of server communication, mechanics of caching, logic of parsing model data, etc. should be split out.

  13. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -62,7 +62,7 @@ So, what are the big picture takeways here? It'll be great to get some more eye

    2. It seems like as the state atom grows, the possible state transitions grow more complex as well. It feels to me like even in a project of this size, the enormous benefits of using plain data structures like maps and seqs start to become outweighed by the unbounded transitions that can be performed on them. My first instinct would be to abstract this by creating helper functions that transition parts of the state atom, and then composing those. Or by making a few number of higher-level functions for semantic transitions like `to-logged-out` or `to-project-page`. This codebase has a fair bit of this already, which is very clear and simple to understand. The comment about needing help to simplify this rings true to me, but I'm more concerned about whether the semantics of the state transition are valid (including that they're composed correctly), than using middleware mechanics to change how they are composed. I wonder also if by using the global state atom, we actually make this problem much worse. It'd be interesting to look in more depth another time at how these state transitions work for this specific app, and whether there are natural seams to break this up along (even if they are stored in a single global atom). These seams might be along either the entities that are present (e.g., "transition between there being a current project or not"), or they might be along the pages themselves (e.g., "transition to state for dominant-component X").

    3. Similarly, the architecture lines for the channels and the controllers (e.g., navigation/API/controls/errors) feels forced, and like they are organizing parts of the app based on the "what" or "how" rather than the "why". As an example, the code for responding to most API calls is organized into a single `api` module. The context of the "why" of the return value, or what this code is doing in the context of the app or the user experience is lost. The best place to see the "why" is in the navigation controller for the `:dashboard`. But it's in a `navigation` module, mixed with the "why" of the other pages. It seems much more natural to me that the holistic higher-level "why" should be fully described in the `dashboard` component. That component might make use of common `api` or `navigation` code bits to perform its work, or split out the mechanics and "how" into separate modules. But it feels to me like the important semantic parts, the meaning and flow of the product, should be expressed directly and wholly in one place. My instinct is that the "dashboard" component is actually the right place for this.
    3. Similarly, the architecture lines for the channels and the controllers (e.g., navigation/API/controls/errors) feel a bit forced, like they are organizing parts of the app based on the "what" or "how" rather than the "why". As an example, the code for responding to most API calls is organized into a single `api` module. The context of the "why" of the return value, or what this code is doing in the context of the app or the user experience is lost. The best place to see the "why" is in the navigation controller for the `:dashboard`. But it's in a `navigation` module, mixed with the "why" of the other pages. It seems much more natural to me that the holistic higher-level "why" should be fully described in the `dashboard` component. That component might make use of common `api` or `navigation` code bits to perform its work, or split out the mechanics and "how" into separate modules. But it feels to me like the important semantic parts, the meaning and flow of the product, should be expressed directly and wholly in one place. My instinct is that the "dashboard" component is actually the right place for this.

    4. Continuing further, moving the orchestration of API requests into a component gives it a natural lifecycle for avoiding race conditions, even with a global state atom. For example, when the `dashboard` component is unmounted, any of its requests could be aborted. The callback code for updating the state atom can be owned and contained within the component's lifecycle. This provides the kind of guard that's accomplished now by manual checks in the API controller, and in a more natural way. As the components grow more complex, the mechanics for this are the bits of code that are split out to keep the complexity low. But by divorcing the architecture from the product and UX (which can only be encoded in View components), we actually make the problem of making new products and features harder. These things shouldn't be decoupled, only the "how" details of server communication, mechanics of caching, logic of parsing model data, etc. should be split out.

  14. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -60,7 +60,7 @@ So, what are the big picture takeways here? It'll be great to get some more eye

    1. This project is amazing, and has a lot of really creative and powerful ideas to learn from! The pervasive use of pure functions leads to being able to reason about the program with only local knowlege in most of the app. Almost all rendering code is isolated from the code that changes app state (thanks React!), which has the enormous benefit of letting us mostly ignore time in rendering code. The functional idea of "data structures first" is really clear here, with pure static functions parsing, interpreting and decorating them. The organization of the model module and functions is intuitive and natural, with any behaviors or processes on that data defined separately. Using pub-sub points as the way for view code to send events into the rest of the app is really clear, and the mechanism of `handler(e)` -> `"update state in place" and "perform other effects like making requests"` makes it straightforward to reason about these state changes in isolation.

    2. It seems like as the state atom grows, the possible state transitions grow more complex as well. It feels to me like even in a project of this size, the enormous benefits of using plain data structures like maps and seqs start to become outweighed by the unbounded transitions that can be performed on them. My first instinct would be to abstract this by creating helper functions that transition parts of the state atom, and then composing those. Or by making a few number of higher-level functions for semantic transitions like `to-logged-out` or `to-project-page`. This codebase has a fair bit of this already, which is very clear and simple to understand. The comment about needing help to simplify this rings true to me, but I'm more concerned about whether the semantics of the state transition are valid (including that they're composed correctly), than using middleware mechanics to change how they are composed. I wonder also if by using the global state atom, we actually make this problem much worse. It'd be another interesting thing to look at how these state transitions work for this specific app, and whether there are natural seams along either the structure of the atom or the use of the data in the atom by `pages` within the view.
    2. It seems like as the state atom grows, the possible state transitions grow more complex as well. It feels to me like even in a project of this size, the enormous benefits of using plain data structures like maps and seqs start to become outweighed by the unbounded transitions that can be performed on them. My first instinct would be to abstract this by creating helper functions that transition parts of the state atom, and then composing those. Or by making a few number of higher-level functions for semantic transitions like `to-logged-out` or `to-project-page`. This codebase has a fair bit of this already, which is very clear and simple to understand. The comment about needing help to simplify this rings true to me, but I'm more concerned about whether the semantics of the state transition are valid (including that they're composed correctly), than using middleware mechanics to change how they are composed. I wonder also if by using the global state atom, we actually make this problem much worse. It'd be interesting to look in more depth another time at how these state transitions work for this specific app, and whether there are natural seams to break this up along (even if they are stored in a single global atom). These seams might be along either the entities that are present (e.g., "transition between there being a current project or not"), or they might be along the pages themselves (e.g., "transition to state for dominant-component X").

    3. Similarly, the architecture lines for the channels and the controllers (e.g., navigation/API/controls/errors) feels forced, and like they are organizing parts of the app based on the "what" or "how" rather than the "why". As an example, the code for responding to most API calls is organized into a single `api` module. The context of the "why" of the return value, or what this code is doing in the context of the app or the user experience is lost. The best place to see the "why" is in the navigation controller for the `:dashboard`. But it's in a `navigation` module, mixed with the "why" of the other pages. It seems much more natural to me that the holistic higher-level "why" should be fully described in the `dashboard` component. That component might make use of common `api` or `navigation` code bits to perform its work, or split out the mechanics and "how" into separate modules. But it feels to me like the important semantic parts, the meaning and flow of the product, should be expressed directly and wholly in one place. My instinct is that the "dashboard" component is actually the right place for this.

  15. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -58,7 +58,7 @@ Discussion

    So, what are the big picture takeways here? It'll be great to get some more eyes on this and verify my understanding, but I'm going to reflect a bit, optimistically assuming that it's at least roughly correct. :)

    1. This project is amazing, and has a lot of really creative and powerful ideas to learn from! The pervasive use of pure functions leads to being able to reason about the program with only local knowlege in most of the app. Almost all rendering code is isolated from the code that changes app state (thanks React!), which has the enormous benefit of letting us mostly ignore time in rendering code. The organization of the model module and functions is intuitive and natural, with any behaviors or processes on that data defined separately. Using pub-sub points as the way for view code to send events into the rest of the app is really clear, and the mechanism of `handler(e)` -> `"update state in place" and "perform other effects like making requests"` makes it straightforward to reason about these state changes in isolation. The functional idea of "data structures first" is really clear here, with pure static functions parsing, interpreting and decorating them.
    1. This project is amazing, and has a lot of really creative and powerful ideas to learn from! The pervasive use of pure functions leads to being able to reason about the program with only local knowlege in most of the app. Almost all rendering code is isolated from the code that changes app state (thanks React!), which has the enormous benefit of letting us mostly ignore time in rendering code. The functional idea of "data structures first" is really clear here, with pure static functions parsing, interpreting and decorating them. The organization of the model module and functions is intuitive and natural, with any behaviors or processes on that data defined separately. Using pub-sub points as the way for view code to send events into the rest of the app is really clear, and the mechanism of `handler(e)` -> `"update state in place" and "perform other effects like making requests"` makes it straightforward to reason about these state changes in isolation.

    2. It seems like as the state atom grows, the possible state transitions grow more complex as well. It feels to me like even in a project of this size, the enormous benefits of using plain data structures like maps and seqs start to become outweighed by the unbounded transitions that can be performed on them. My first instinct would be to abstract this by creating helper functions that transition parts of the state atom, and then composing those. Or by making a few number of higher-level functions for semantic transitions like `to-logged-out` or `to-project-page`. This codebase has a fair bit of this already, which is very clear and simple to understand. The comment about needing help to simplify this rings true to me, but I'm more concerned about whether the semantics of the state transition are valid (including that they're composed correctly), than using middleware mechanics to change how they are composed. I wonder also if by using the global state atom, we actually make this problem much worse. It'd be another interesting thing to look at how these state transitions work for this specific app, and whether there are natural seams along either the structure of the atom or the use of the data in the atom by `pages` within the view.

  16. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -15,7 +15,7 @@ So cheers! I'd welcome any thoughts or feedback you'd be willing to share back.
    Let's go!
    ---

    Ok, the entrypoing is `core/setup!`, called from CoffeeScript.
    Ok, the entrypoint is `core/setup!`, called from CoffeeScript.

    After defining an initial `state`, it creates a new `history-imp`, which will listen for browser history events and will call `sec/dispatch!` as part of starting up. I'm assuming there's an event loop tick before this gets called, since otherwise it looks like `main` and `define-routes!` won't have been called yet.

  17. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -2,7 +2,7 @@ Hello!

    This is a commentary and discussion from walking through what happens on startup in [https://github.com/circleci/frontend], looking at this SHA in December 2014: [https://github.com/circleci/frontend/commit/273558250040ce197e44e683d5528381f36c4eea].

    The first part is a literal walkthrough, where I'm tracing my reading of how the code works. This is a way to check my understanding, since of course I don't have all of your context. And it's also a way to echo back to you how someone else might experience this code without that context. The second part is about asking questions, making suggestions, and exploring some paths for making it even more awesome. I also mixed in a few asides in the first part where I jump ahead a bit.
    The first part is a literal walkthrough, where I'm tracing my reading of how the code works. This is a way to check my understanding, since of course I don't have all of your context. And it's also a way to echo back to you how someone else might experience this code without that context. The second part is about asking questions, making suggestions, and exploring some paths for making it even more awesome. I also mixed in a few asides in the walkthrough part where I jump ahead a bit.

    Doing this with colleagues has worked well for me as a way to do in-depth code reviews about more abstract design and architectural questions. The goal of the the first part is to clarify we're all looking at the same thing, which creates the space and shared understanding to do a good critique in the second part where we both learn things. :)

  18. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -2,7 +2,7 @@ Hello!

    This is a commentary and discussion from walking through what happens on startup in [https://github.com/circleci/frontend], looking at this SHA in December 2014: [https://github.com/circleci/frontend/commit/273558250040ce197e44e683d5528381f36c4eea].

    The first part is a literal walkthrough, where I'm tracing my reading of how the code works. This is a way to check my understanding, since of course I don't have all of your context. And it's also a way to echo back to you how someone else might experience this code without that context. The second part is about asking questions, making suggestions, and trying to learn together about how well the current design fits the product or application, and explore some paths for making it even more awesome.
    The first part is a literal walkthrough, where I'm tracing my reading of how the code works. This is a way to check my understanding, since of course I don't have all of your context. And it's also a way to echo back to you how someone else might experience this code without that context. The second part is about asking questions, making suggestions, and exploring some paths for making it even more awesome. I also mixed in a few asides in the first part where I jump ahead a bit.

    Doing this with colleagues has worked well for me as a way to do in-depth code reviews about more abstract design and architectural questions. The goal of the the first part is to clarify we're all looking at the same thing, which creates the space and shared understanding to do a good critique in the second part where we both learn things. :)

  19. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -2,7 +2,7 @@ Hello!

    This is a commentary and discussion from walking through what happens on startup in [https://github.com/circleci/frontend], looking at this SHA in December 2014: [https://github.com/circleci/frontend/commit/273558250040ce197e44e683d5528381f36c4eea].

    The first part is a literal walkthrough, where I'm tracing my reading of how the code works. This is a way to check my understanding, since of course I don't have all of your context. And it's also a way to echo back to you how someone else might experience this code without that context. The second part is about asking questions, making suggestions, and having a conversation that, might help us learn something about how well the design here fits the product or application and paths for making it even more awesome.
    The first part is a literal walkthrough, where I'm tracing my reading of how the code works. This is a way to check my understanding, since of course I don't have all of your context. And it's also a way to echo back to you how someone else might experience this code without that context. The second part is about asking questions, making suggestions, and trying to learn together about how well the current design fits the product or application, and explore some paths for making it even more awesome.

    Doing this with colleagues has worked well for me as a way to do in-depth code reviews about more abstract design and architectural questions. The goal of the the first part is to clarify we're all looking at the same thing, which creates the space and shared understanding to do a good critique in the second part where we both learn things. :)

  20. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -2,7 +2,7 @@ Hello!

    This is a commentary and discussion from walking through what happens on startup in [https://github.com/circleci/frontend], looking at this SHA in December 2014: [https://github.com/circleci/frontend/commit/273558250040ce197e44e683d5528381f36c4eea].

    The first part is a literal walkthrough, where I'm tracing my reading of how the code works. This is a way to check my understanding, since of course I don't have all of your context. And it's also a way to echo back to you how someone else might experience this code without that context. The second part is about asking questions, making suggestions, and having a conversation that, over time, will help us sync on our understanding of how well the design here fits the product or application.
    The first part is a literal walkthrough, where I'm tracing my reading of how the code works. This is a way to check my understanding, since of course I don't have all of your context. And it's also a way to echo back to you how someone else might experience this code without that context. The second part is about asking questions, making suggestions, and having a conversation that, might help us learn something about how well the design here fits the product or application and paths for making it even more awesome.

    Doing this with colleagues has worked well for me as a way to do in-depth code reviews about more abstract design and architectural questions. The goal of the the first part is to clarify we're all looking at the same thing, which creates the space and shared understanding to do a good critique in the second part where we both learn things. :)

  21. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,6 @@
    Hello!

    This is a walkthough of what happens on startup in [https://github.com/circleci/frontend], looking at this SHA in December 2014: [https://github.com/circleci/frontend/commit/273558250040ce197e44e683d5528381f36c4eea].
    This is a commentary and discussion from walking through what happens on startup in [https://github.com/circleci/frontend], looking at this SHA in December 2014: [https://github.com/circleci/frontend/commit/273558250040ce197e44e683d5528381f36c4eea].

    The first part is a literal walkthrough, where I'm tracing my reading of how the code works. This is a way to check my understanding, since of course I don't have all of your context. And it's also a way to echo back to you how someone else might experience this code without that context. The second part is about asking questions, making suggestions, and having a conversation that, over time, will help us sync on our understanding of how well the design here fits the product or application.

  22. kevinrobinson renamed this gist Jan 2, 2015. 1 changed file with 0 additions and 14 deletions.
    14 changes: 0 additions & 14 deletions walkthrough.md → circleci-frontend-walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -1,17 +1,3 @@
    Brendon,

    My name's Kevin Robinson, and I'm super excited about your work!

    I read through the CircleCI frontend repo in some depth, and wrote up some notes and thoughts on it. I'm hoping you'll be open to reading it through and sharing your thoughts. Even if not, your work has been tremendously valuable and I'm grateful you're sharing it in blog posts and as open source. :)

    Here's a gist:

    -Kevin
    [email protected]
    @krob



    Hello!

    This is a walkthough of what happens on startup in [https://github.com/circleci/frontend], looking at this SHA in December 2014: [https://github.com/circleci/frontend/commit/273558250040ce197e44e683d5528381f36c4eea].
  23. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 9 additions and 9 deletions.
    18 changes: 9 additions & 9 deletions walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -72,33 +72,33 @@ Discussion

    So, what are the big picture takeways here? It'll be great to get some more eyes on this and verify my understanding, but I'm going to reflect a bit, optimistically assuming that it's at least roughly correct. :)

    1. This project is amazing, and has a lot of really creative and powerful ideas to learn from! The pervasive use of pure functions leads to being able to reason about the program with only local knowlege in most of the app. Almost all rendering code is isolated from the code that changes app state (thanks React!), which has the enormous benefit of letting us mostly ignore time in rendering code. Using pub-sub points as the way for view code to send events into the rest of the app is really clear, and the mechanism of `handler(e)` -> `"update state in place" and "perform other effects like making requests"` makes it straightforward to reason about these state changes in isolation. I'm fully sold on the functional idea of `data structures first`, with pure static functions parsing, interpreting and decorating them. I love the way the models functions are split out here.
    1. This project is amazing, and has a lot of really creative and powerful ideas to learn from! The pervasive use of pure functions leads to being able to reason about the program with only local knowlege in most of the app. Almost all rendering code is isolated from the code that changes app state (thanks React!), which has the enormous benefit of letting us mostly ignore time in rendering code. The organization of the model module and functions is intuitive and natural, with any behaviors or processes on that data defined separately. Using pub-sub points as the way for view code to send events into the rest of the app is really clear, and the mechanism of `handler(e)` -> `"update state in place" and "perform other effects like making requests"` makes it straightforward to reason about these state changes in isolation. The functional idea of "data structures first" is really clear here, with pure static functions parsing, interpreting and decorating them.

    2. It seems like as the state atom grows, the possible state transitions grow enormously as well. It feels to me like even in a project of this size, the enormous benefits of using plain data structures like maps and seqs start to become outweighed by the unbounded transitions that can be performed on them. In practice, I imagine this would be abstracted by creating helper functions that transition parts of the state atom and can be composed, or by making a few number of higher-level functions for semantic transitions like `to-logged-out` or `to-project-page`. This codebase has a fair bit of this already, which is very clear and simple to understand. The comment about needing help to simplify this rings true to me, but I'm more concerned about whether the semantics of the state transition are valid (including that they're composed correctly). I wonder also if by using the global state atom, we actually make this problem much worse. It'd be another interesting thing to look at how these state transitions work for this specific app, and whether there are natural seams along either the structure of the atom or the use of the data in the atom by `pages` within the view.
    2. It seems like as the state atom grows, the possible state transitions grow more complex as well. It feels to me like even in a project of this size, the enormous benefits of using plain data structures like maps and seqs start to become outweighed by the unbounded transitions that can be performed on them. My first instinct would be to abstract this by creating helper functions that transition parts of the state atom, and then composing those. Or by making a few number of higher-level functions for semantic transitions like `to-logged-out` or `to-project-page`. This codebase has a fair bit of this already, which is very clear and simple to understand. The comment about needing help to simplify this rings true to me, but I'm more concerned about whether the semantics of the state transition are valid (including that they're composed correctly), than using middleware mechanics to change how they are composed. I wonder also if by using the global state atom, we actually make this problem much worse. It'd be another interesting thing to look at how these state transitions work for this specific app, and whether there are natural seams along either the structure of the atom or the use of the data in the atom by `pages` within the view.

    3. Similarly, the architecture lines for API and navigation feels forced, and like they are organizing parts of the app based on the "what" or "how" rather than the "why". As an example, the code for responding to most API calls is organized into a single `api` module. The context of the "why" of the return value, or what this code is doing in the context of the app or the user experience is lost. The best place to see the "why" is in the navigation controller for the `:dashboard`. But it's in a `navigation` module, mixed with the "why" of the other pages. It seems much more natural to me that the holistic higher-level "why" should be fully described in the `dashboard` component. That component might make use of common `api` or `navigation` code bits to perform its work, or split out the mechanics and "how" into separate modules. But it feels to me like the important semantic parts, the meaning and flow of the product, should be expressed directly and wholly in one place. My instinct is that the "dashboard" component is actually the right place for this.
    3. Similarly, the architecture lines for the channels and the controllers (e.g., navigation/API/controls/errors) feels forced, and like they are organizing parts of the app based on the "what" or "how" rather than the "why". As an example, the code for responding to most API calls is organized into a single `api` module. The context of the "why" of the return value, or what this code is doing in the context of the app or the user experience is lost. The best place to see the "why" is in the navigation controller for the `:dashboard`. But it's in a `navigation` module, mixed with the "why" of the other pages. It seems much more natural to me that the holistic higher-level "why" should be fully described in the `dashboard` component. That component might make use of common `api` or `navigation` code bits to perform its work, or split out the mechanics and "how" into separate modules. But it feels to me like the important semantic parts, the meaning and flow of the product, should be expressed directly and wholly in one place. My instinct is that the "dashboard" component is actually the right place for this.

    4. Continuing further, moving the orchestration of API requests into a component gives it a natural lifecycle for avoiding race conditions, even with a global state atom. For example, when the `dashboard` component is unmounted, any of its requests could be aborted. The callback code for updating the state atom can be owned and contained within the component's lifecycle. This provides the kind of guard that's accomplished now by manual checks in the API controller, and in a more natural way. As the components grow more complex, the mechanics for this are the bits of code that are split out to keep the complexity low. But by divorcing the architecture from the product and UX (which can only be encoded in View components), we actually make the problem of making new products and features harder. These things shouldn't be decoupled, only the "how" details of server communication, mechanics of caching, logic of parsing model data, etc. should be split out.

    5. Using channels for tracking server communication feel like a mismatch for the problem here. This isn't about their local use, but about putting events about emitting events about state changes for requests on a pub-sub point. Using the example of the `navigation` controller's making calls for the `dashboard`, it creates a parallel kind of application state that isn't made concrete in the state atom. The application state can't actually be entirely described by the state atom, there's also hidden state with in-flight server requests, and even further, goroutines that controller code has queued up, waiting to perform additional requests and state transitions when the server requests do complete. And there's no way to abort in-flight requests or scheduled effects based other state changes that occur. As an example, after the `:dashboard` navigation event is emitted, let's say the user navigates somewhere else. The first dashboard request is still in flight, and when it completes, there is code already queued up to respond to that and make additional requests. These requests are irrelevant now (the user has told us they don't care about that data anymore), but there's no way to abort those queued effects. You can see this complexity leaking into the API controller code with those guards. Note: I'm not arguing against channels in general, in fact I think the mechanics or Promises/callbacks/channels are not particularly important. In the way channels are used locally, like with `:get builds` in the navigation controller, it's not any different than using Promises or callbacks, that's cool. If we needed some more complex composition, local use of channels and goroutines seems awesome (although I haven't yet seen a compelling real-world use case for channels over Promises, other than the Golang example of fanning out and taking the fastest responders, and that doesn't translate well to the browser). My point here is about the hidden state of in-flight server requests, and code that's queued up to run, neither of which are encoded in the state atom.

    Takeaway
    ---
    This code is spectacularly well-crafted and reading it has been a fanstastic learning experience. I imagine I'll keep following the work you all are doing, as in my mind you're doing some of the leading creative work in this area, and the kind of work that has the potential to really push us forward and help the whole field.
    This code is spectacularly well-crafted and reading it has been a fantastic learning experience. I imagine I'll keep following the work you are doing, as in my mind you're doing some of the leading creative work in this area, and the kind of work that has the potential to really push us forward and help the whole field.

    My instinct is that all front-end architectures are missing the perspective of what's really important; flexibilty and speed with adding new product features, and limiting coupling between distint product features as a way to avoid premature abstraction. But reducing coupling isn't an abstract goal, it's in service of being able to make new features more quickly, or change existing code to meet requirements that were unknown at the time. Refactoring is easy; re-abstracting is difficult.
    My instinct is that all front-end architectures are missing the perspective of orienting and evaluating themselves on what's really important: flexibilty and speed with adding new product features, and limiting coupling between distint product features as a way to avoid premature abstraction. But reducing coupling isn't an abstract goal, it's in service of being able to make new features more quickly, or change existing code to meet requirements that were unknown at the time. Refactoring is easy and mechanical; re-abstracting is difficult and requires deep understanding.

    I think the process and dynamic parts of UI code is where the hard parts are that plague us most day-to-day. And taking the strengths of CSP to model dynamic processes, and applying them to UIs isn't a great fit, since writing correct concurrent systems is a different kind of work, and in my experience, not the kind of work that usually has the best payoff for engineers that live close to the product and user experience.
    I think the process and dynamic parts of UI code is where the hard parts are that plague us most day-to-day. And taking the strengths of CSP to model dynamic processes, and applying them to UIs isn't a great fit, since writing correct concurrent systems is a different kind of work, and in my experience, not the kind of work that usually has the best product payoff for engineers that live close to the product and user experience.

    The other parts of UI programming that I think is truly difficult are the true distributed systems problems of data consistency. But in practice, most applications don't actually need to solve this, or the investment isn't worth it. Simpler approximations are almost always sufficient. Like taking a snapshot of the server state and holding it as immutable for a period of seconds or minutes, predicting that the likelihook of it changing underneath the user is very low and the path to recovery when it does happen is very straightforward. Or embracing eventual consistency and decoupling, and allowing data to be updated in different views within seconds of each other rather than synchronously. Or by investing in improvements to the API server layer so that server response times are reliably within 100s of milliseconds and more extensive browser/device-side caching is not worth the complexity.
    The other parts of UI programming that I think are truly difficult are the true distributed systems problems of data consistency. But in practice, most applications don't actually need to solve this, or the engineering investment isn't worth it for the product payoff. Simpler approximations are almost always sufficient. Like taking a snapshot of the server state and holding it as immutable for a period of seconds or minutes, predicting that the likelihook of it changing underneath the user is very low and the path to recovery when it does happen is very straightforward. Or embracing eventual consistency and decoupling, and allowing data to be updated in different views within seconds of each other rather than synchronously. Or by investing in improvements to the API serving layer so that server response times are reliably within 100s of milliseconds and more extensive browser/device-side caching is not worth the complexity.

    And if we do want to make a product or feature that functions as a true distributed system, that of all things should be split out from the rest of the application. :) The idea that feels the most promising is making the "view of server state" immutable, and, within the application, recording facts in the style of Datomic about server state and changes in client state. This way view code can query against that, merging optimistic client updates with the view of server state, retracting optimistic updates that the server later rejects, etc.
    But if we do want to make a product or feature that functions as a true distributed system, that of all things should be split out from the rest of the application. :) The idea that feels the most promising is making the "view of server state" immutable, and, within the application, recording facts about server state and changes in client state. This way view code can compute views of those facts, like merging optimistic client updates with the view of server state. And there's a first-class way to deal with failure, like retracting facts about optimistic updates that the server later rejects, etc. I think the real architectural achievement is when this design is simple enough to work with that these guidelines still seem natural, intuitive, and unburdensome for small forms or chat apps.

    So my closing two thoughts are:

    1. What if we reified effects, like in-flight server requests, into state? And then the function that took prev-state -> new-state could consider the state of effects as part of producing the new state value, and it could also perform effects as needed (e.g., abort in-flight requests that aren't relevant anymore, make requests for additional data, kick off another iteration of a simulation).

    2. The global state atom is being mutated in-place. That works well for the representation of the UI state, since the DOM ultimately is a place that must be mutated. But it doesn't work as well for a sophisticated client-side view of server state. So what if split these, and wrote facts about the client-side view of server into an in-browser database in the style of Datomic? Components could d/transact or d/query database values, and recompute the views they need on database changes. I think the real architectural achievement is when this is simple enough that for small forms or chat apps, that these guidelines still seem natural and intuitive.
    2. The global state atom is being mutated in-place. That works well for the representation of the UI state, since the DOM ultimately is a place that must be mutated. But it doesn't work as well for a sophisticated client-side view of server state. What if we split out the full client-side view of server state (including optimistic updates) to an immutable data store in the browser?

    Of course, I'm writing you because I admire your work tremendously, and would really value any feedback or thoughts you had the time to share. Keep doing (and sharing) your awesome work! :)

  24. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 18 additions and 8 deletions.
    26 changes: 18 additions & 8 deletions walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -70,26 +70,36 @@ Ok, so that's about it for that code path. One more note before popping up the
    Discussion
    ---

    So, what are the big picture takeways here? It'll be great to get some more eyes on this and verify my understanding, but I'm going to reflect a bit assuming this is roughly correct.
    So, what are the big picture takeways here? It'll be great to get some more eyes on this and verify my understanding, but I'm going to reflect a bit, optimistically assuming that it's at least roughly correct. :)

    1. This project is amazing, and has a lot of really creative and powerful ideas to learn from.
    1. This project is amazing, and has a lot of really creative and powerful ideas to learn from! The pervasive use of pure functions leads to being able to reason about the program with only local knowlege in most of the app. Almost all rendering code is isolated from the code that changes app state (thanks React!), which has the enormous benefit of letting us mostly ignore time in rendering code. Using pub-sub points as the way for view code to send events into the rest of the app is really clear, and the mechanism of `handler(e)` -> `"update state in place" and "perform other effects like making requests"` makes it straightforward to reason about these state changes in isolation. I'm fully sold on the functional idea of `data structures first`, with pure static functions parsing, interpreting and decorating them. I love the way the models functions are split out here.

    2. Channels feel like a mismatch for the problem here. This isn't about their local use (see footnote #1), but about their use as a pub-sub point for results of server calls and other operations. Using the example of the `navigation` controller's making calls for the `dashboard`, it creates a parallel kind of application state that isn't made concrete in the state atom. The application state can't actually be entirely described by the state atom, there's a lot of hidden state with in-flight server requests, and even further, goroutines that have code queued up, waiting to perform additional requests and state transitions when the server requests do complete. You can see this complexity leaking into the API controller code with those guards. Note: I'm not arguing against channels in general, in fact I think the mechanics or Promises/callbacks/channels are not particularly important. In the way channels are used locally, like with `:get builds` in the navigation controller, it's not any different than using Promises or callbacks, that's cool. If we needed some more complex composition, local use of channels and goroutines seems awesome (although I haven't yet seen a compelling real-world use case for channels over Promises, other than the Golang example of fanning out and taking the fastest responders, and that doesn't translate well to the browser). My point here is about the hidden state of in-flight server requests, and code that's queued up to run, neither of which are encoded in the state atom.
    2. It seems like as the state atom grows, the possible state transitions grow enormously as well. It feels to me like even in a project of this size, the enormous benefits of using plain data structures like maps and seqs start to become outweighed by the unbounded transitions that can be performed on them. In practice, I imagine this would be abstracted by creating helper functions that transition parts of the state atom and can be composed, or by making a few number of higher-level functions for semantic transitions like `to-logged-out` or `to-project-page`. This codebase has a fair bit of this already, which is very clear and simple to understand. The comment about needing help to simplify this rings true to me, but I'm more concerned about whether the semantics of the state transition are valid (including that they're composed correctly). I wonder also if by using the global state atom, we actually make this problem much worse. It'd be another interesting thing to look at how these state transitions work for this specific app, and whether there are natural seams along either the structure of the atom or the use of the data in the atom by `pages` within the view.

    3. It seems like as the state atom grows, the possible state transitions grow enormously as well. It feels to me like even in a project of this size, the enormous benefits of using plain data structures like maps and seqs start to become outweighed by the unbounded transitions that can be performed on them. In practice, I imagine this would be abstracted by creating helper functions that transition parts of the state atom and can be composed, or by making a few number of higher-level functions for semantic transitions like `to-logged-out` or `to-project-page`. This codebase has a fair bit of this already, but the comment about needing help to simplify this rings true to me. I wonder also if by forcing the global state atom, we actually make this problem much worse. It'd be another interesting thing to look at how these state transitions work for this specific app, and whether there are natural seams along either the structure of the atom or the use of the data in the atom by `pages` within the view.
    3. Similarly, the architecture lines for API and navigation feels forced, and like they are organizing parts of the app based on the "what" or "how" rather than the "why". As an example, the code for responding to most API calls is organized into a single `api` module. The context of the "why" of the return value, or what this code is doing in the context of the app or the user experience is lost. The best place to see the "why" is in the navigation controller for the `:dashboard`. But it's in a `navigation` module, mixed with the "why" of the other pages. It seems much more natural to me that the holistic higher-level "why" should be fully described in the `dashboard` component. That component might make use of common `api` or `navigation` code bits to perform its work, or split out the mechanics and "how" into separate modules. But it feels to me like the important semantic parts, the meaning and flow of the product, should be expressed directly and wholly in one place. My instinct is that the "dashboard" component is actually the right place for this.

    4. The architecture lines for API and navigation feels forced, and like they are organizing parts of the app based on the "what" or "how" rather than the "why". As an example, the code for responding to most API calls is organized into a single `api` module. The context of the "why" of the return value, or what this code is doing in the context of the app or the user experience is lost. The best place to see the "why" is in the navigation controller for the :dashboard. But it's in a `navigation` module, mixed with the "why" of the other pages. It seems much more natural to me that the holistic higher-level "why" should live together in the `dashboard` component. It might make use of common `api` or `navigation` code bits to perform its work, but the important semantic parts, the meaning and flow of the product, feel like they should live together in the "dashboard" component, or right off it.
    4. Continuing further, moving the orchestration of API requests into a component gives it a natural lifecycle for avoiding race conditions, even with a global state atom. For example, when the `dashboard` component is unmounted, any of its requests could be aborted. The callback code for updating the state atom can be owned and contained within the component's lifecycle. This provides the kind of guard that's accomplished now by manual checks in the API controller, and in a more natural way. As the components grow more complex, the mechanics for this are the bits of code that are split out to keep the complexity low. But by divorcing the architecture from the product and UX (which can only be encoded in View components), we actually make the problem of making new products and features harder. These things shouldn't be decoupled, only the "how" details of server communication, mechanics of caching, logic of parsing model data, etc. should be split out.

    5. Continuing further, moving the API requests into a component gives them a natural lifecycle for avoiding race conditions even with a global state atom. For example, when the `dashboard` component is unmounted, any of its requests could be aborted. The callback code for updating the state atom can be owned and contained within the componen'ts lifecycle. This provides the kind of guard that's accomplished now by manual checks in the API controller, and in a more natural way. As the components grow more complex, these are the bits of code that are split out to keep the complexity low. But by divorcing the architecture from the product and UX that can only be encoded in View components, we actually make the problem of making new products and features harder. These things shouldn't be decoupled, only the details of server communication, mechanics of caching, logic of parsing model data, etc. should be split out.
    5. Using channels for tracking server communication feel like a mismatch for the problem here. This isn't about their local use, but about putting events about emitting events about state changes for requests on a pub-sub point. Using the example of the `navigation` controller's making calls for the `dashboard`, it creates a parallel kind of application state that isn't made concrete in the state atom. The application state can't actually be entirely described by the state atom, there's also hidden state with in-flight server requests, and even further, goroutines that controller code has queued up, waiting to perform additional requests and state transitions when the server requests do complete. And there's no way to abort in-flight requests or scheduled effects based other state changes that occur. As an example, after the `:dashboard` navigation event is emitted, let's say the user navigates somewhere else. The first dashboard request is still in flight, and when it completes, there is code already queued up to respond to that and make additional requests. These requests are irrelevant now (the user has told us they don't care about that data anymore), but there's no way to abort those queued effects. You can see this complexity leaking into the API controller code with those guards. Note: I'm not arguing against channels in general, in fact I think the mechanics or Promises/callbacks/channels are not particularly important. In the way channels are used locally, like with `:get builds` in the navigation controller, it's not any different than using Promises or callbacks, that's cool. If we needed some more complex composition, local use of channels and goroutines seems awesome (although I haven't yet seen a compelling real-world use case for channels over Promises, other than the Golang example of fanning out and taking the fastest responders, and that doesn't translate well to the browser). My point here is about the hidden state of in-flight server requests, and code that's queued up to run, neither of which are encoded in the state atom.

    In summary, this code is spectacularly well-crafted and reading it has been a fanstastic learning experience. I imagine I'll keep following the work you all are doing, as in my mind you're doing some of the leading creative work in this area, and the kind of work that has the potential to really push us forward and help the whole field.
    Takeaway
    ---
    This code is spectacularly well-crafted and reading it has been a fanstastic learning experience. I imagine I'll keep following the work you all are doing, as in my mind you're doing some of the leading creative work in this area, and the kind of work that has the potential to really push us forward and help the whole field.

    My instinct is that all front-end architectures are missing the perspective of what's really important; flexibilty and speed with adding new product features, and limiting coupling between distint product features as a way to avoid premature abstraction. But reducing coupling isn't an abstract goal, it's in service of being able to make new features more quickly, or change existing code to meet requirements that were unknown at the time. Refactoring is easy; re-abstracting is difficult.

    I'm fully sold on the functional idea of `data structures first`, with pure static functions parsing, interpreting and decorating them. I love the way the models functions are split out here. But I think the process and dynamic parts of UI code is where the hard parts are that plague us most day-to-day.
    I think the process and dynamic parts of UI code is where the hard parts are that plague us most day-to-day. And taking the strengths of CSP to model dynamic processes, and applying them to UIs isn't a great fit, since writing correct concurrent systems is a different kind of work, and in my experience, not the kind of work that usually has the best payoff for engineers that live close to the product and user experience.

    The other parts of UI programming that I think is truly difficult are the true distributed systems problems of data consistency. But in practice, most applications don't actually need to solve this, or the investment isn't worth it. Simpler approximations are almost always sufficient. Like taking a snapshot of the server state and holding it as immutable for a period of seconds or minutes, predicting that the likelihook of it changing underneath the user is very low and the path to recovery when it does happen is very straightforward. Or embracing eventual consistency and decoupling, and allowing data to be updated in different views within seconds of each other rather than synchronously. Or by investing in improvements to the API server layer so that server response times are reliably within 100s of milliseconds and more extensive browser/device-side caching is not worth the complexity.

    And if we do want to make a product or feature that functions as a true distributed system, that of all things should be split out from the rest of the application. :) The idea that feels the most promising is making the "view of server state" immutable, and, within the application, recording facts in the style of Datomic about server state and changes in client state. This way view code can query against that, merging optimistic client updates with the view of server state, retracting optimistic updates that the server later rejects, etc.

    So my closing two thoughts are:

    1. What if we reified effects, like in-flight server requests, into state? And then the function that took prev-state -> new-state could consider the state of effects as part of producing the new state value, and it could also perform effects as needed (e.g., abort in-flight requests that aren't relevant anymore, make requests for additional data, kick off another iteration of a simulation).

    2. The global state atom is being mutated in-place. That works well for the representation of the UI state, since the DOM ultimately is a place that must be mutated. But it doesn't work as well for a sophisticated client-side view of server state. So what if split these, and wrote facts about the client-side view of server into an in-browser database in the style of Datomic? Components could d/transact or d/query database values, and recompute the views they need on database changes. I think the real architectural achievement is when this is simple enough that for small forms or chat apps, that these guidelines still seem natural and intuitive.

    Of course, I'm writing you because I admire your work tremendously, and would really value any feedback or thoughts you had the time to share. Keep doing (and sharing) your awesome work! :)

    -Kevin
  25. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 6 additions and 5 deletions.
    11 changes: 6 additions & 5 deletions walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -67,19 +67,20 @@ I can't quite follow where the results of the `:project-settings` server call is

    Ok, so that's about it for that code path. One more note before popping up the stack. There's also the `ws-ch`, and possible that there are other paths for state changes that are related to projects and this code path that can happen through messages from the server that get put in that channel. I'll ignore that for now, but keeping of track of in-flight server requests when responding to messsages on the `ws-ch` channel seems like another place that might be tricky.

    Discussion
    ---

    So, what are the big picture takeways here? It'll be great to get some more eyes on this and verify my understanding, but I'm going to reflect a bit assuming this is roughly correct.

    1) This project is amazing, and has a lot of really creative and powerful ideas to learn from.
    1. This project is amazing, and has a lot of really creative and powerful ideas to learn from.

    2) Channels feel like a mismatch for the problem here. This isn't about their local use (see footnote #1), but about their use as a pub-sub point for results of server calls and other operations. Using the example of the `navigation` controller's making calls for the `dashboard`, it creates a parallel kind of application state that isn't made concrete in the state atom. The application state can't actually be entirely described by the state atom, there's a lot of hidden state with in-flight server requests, and even further, goroutines that have code queued up, waiting to perform additional requests and state transitions when the server requests do complete. You can see this complexity leaking into the API controller code with those guards. Note: I'm not arguing against channels in general, in fact I think the mechanics or Promises/callbacks/channels are not particularly important. In the way channels are used locally, like with `:get builds` in the navigation controller, it's not any different than using Promises or callbacks, that's cool. If we needed some more complex composition, local use of channels and goroutines seems awesome (although I haven't yet seen a compelling real-world use case for channels over Promises, other than the Golang example of fanning out and taking the fastest responders, and that doesn't translate well to the browser). My point here is about the hidden state of in-flight server requests, and code that's queued up to run, neither of which are encoded in the state atom.
    2. Channels feel like a mismatch for the problem here. This isn't about their local use (see footnote #1), but about their use as a pub-sub point for results of server calls and other operations. Using the example of the `navigation` controller's making calls for the `dashboard`, it creates a parallel kind of application state that isn't made concrete in the state atom. The application state can't actually be entirely described by the state atom, there's a lot of hidden state with in-flight server requests, and even further, goroutines that have code queued up, waiting to perform additional requests and state transitions when the server requests do complete. You can see this complexity leaking into the API controller code with those guards. Note: I'm not arguing against channels in general, in fact I think the mechanics or Promises/callbacks/channels are not particularly important. In the way channels are used locally, like with `:get builds` in the navigation controller, it's not any different than using Promises or callbacks, that's cool. If we needed some more complex composition, local use of channels and goroutines seems awesome (although I haven't yet seen a compelling real-world use case for channels over Promises, other than the Golang example of fanning out and taking the fastest responders, and that doesn't translate well to the browser). My point here is about the hidden state of in-flight server requests, and code that's queued up to run, neither of which are encoded in the state atom.

    3) It seems like as the state atom grows, the possible state transitions grow enormously as well. It feels to me like even in a project of this size, the enormous benefits of using plain data structures like maps and seqs start to become outweighed by the unbounded transitions that can be performed on them. In practice, I imagine this would be abstracted by creating helper functions that transition parts of the state atom and can be composed, or by making a few number of higher-level functions for semantic transitions like `to-logged-out` or `to-project-page`. This codebase has a fair bit of this already, but the comment about needing help to simplify this rings true to me. I wonder also if by forcing the global state atom, we actually make this problem much worse. It'd be another interesting thing to look at how these state transitions work for this specific app, and whether there are natural seams along either the structure of the atom or the use of the data in the atom by `pages` within the view.
    3. It seems like as the state atom grows, the possible state transitions grow enormously as well. It feels to me like even in a project of this size, the enormous benefits of using plain data structures like maps and seqs start to become outweighed by the unbounded transitions that can be performed on them. In practice, I imagine this would be abstracted by creating helper functions that transition parts of the state atom and can be composed, or by making a few number of higher-level functions for semantic transitions like `to-logged-out` or `to-project-page`. This codebase has a fair bit of this already, but the comment about needing help to simplify this rings true to me. I wonder also if by forcing the global state atom, we actually make this problem much worse. It'd be another interesting thing to look at how these state transitions work for this specific app, and whether there are natural seams along either the structure of the atom or the use of the data in the atom by `pages` within the view.

    4) The architecture lines for API and navigation feels forced, and like they are organizing parts of the app based on the "what" or "how" rather than the "why". As an example, the code for responding to most API calls is organized into a single `api` module. The context of the "why" of the return value, or what this code is doing in the context of the app or the user experience is lost. The best place to see the "why" is in the navigation controller for the :dashboard. But it's in a `navigation` module, mixed with the "why" of the other pages. It seems much more natural to me that the holistic higher-level "why" should live together in the `dashboard` component. It might make use of common `api` or `navigation` code bits to perform its work, but the important semantic parts, the meaning and flow of the product, feel like they should live together in the "dashboard" component, or right off it.
    4. The architecture lines for API and navigation feels forced, and like they are organizing parts of the app based on the "what" or "how" rather than the "why". As an example, the code for responding to most API calls is organized into a single `api` module. The context of the "why" of the return value, or what this code is doing in the context of the app or the user experience is lost. The best place to see the "why" is in the navigation controller for the :dashboard. But it's in a `navigation` module, mixed with the "why" of the other pages. It seems much more natural to me that the holistic higher-level "why" should live together in the `dashboard` component. It might make use of common `api` or `navigation` code bits to perform its work, but the important semantic parts, the meaning and flow of the product, feel like they should live together in the "dashboard" component, or right off it.

    5) Continuing further, moving the API requests into a component gives them a natural lifecycle for avoiding race conditions even with a global state atom. For example, when the `dashboard` component is unmounted, any of its requests could be aborted. The callback code for updating the state atom can be owned and contained within the componen'ts lifecycle. This provides the kind of guard that's accomplished now by manual checks in the API controller, and in a more natural way. As the components grow more complex, these are the bits of code that are split out to keep the complexity low. But by divorcing the architecture from the product and UX that can only be encoded in View components, we actually make the problem of making new products and features harder. These things shouldn't be decoupled, only the details of server communication, mechanics of caching, logic of parsing model data, etc. should be split out.
    5. Continuing further, moving the API requests into a component gives them a natural lifecycle for avoiding race conditions even with a global state atom. For example, when the `dashboard` component is unmounted, any of its requests could be aborted. The callback code for updating the state atom can be owned and contained within the componen'ts lifecycle. This provides the kind of guard that's accomplished now by manual checks in the API controller, and in a more natural way. As the components grow more complex, these are the bits of code that are split out to keep the complexity low. But by divorcing the architecture from the product and UX that can only be encoded in View components, we actually make the problem of making new products and features harder. These things shouldn't be decoupled, only the details of server communication, mechanics of caching, logic of parsing model data, etc. should be split out.

    In summary, this code is spectacularly well-crafted and reading it has been a fanstastic learning experience. I imagine I'll keep following the work you all are doing, as in my mind you're doing some of the leading creative work in this area, and the kind of work that has the potential to really push us forward and help the whole field.

  26. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 5 additions and 5 deletions.
    10 changes: 5 additions & 5 deletions walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -47,17 +47,17 @@ In `defmethod navigated-to :dashboard`, (https://github.com/circleci/frontend/bl
    After swapping to the new state, the `nav-handler` calls `post-navigated-to! :dashboard` (https://github.com/circleci/frontend/blob/master/src-cljs/frontend/controllers/navigation.cljs#L91). This does some updates immediately, like `set-page-title!` and recording analytics. And most importantly, it performs several calls to the server and schedules a sequence of them.

    `post-navigated-to! :dashboard` may call `api/get-projects`, and always calls `ajax/managed-ajax :get builds-url)`. In a goroutine, it blocks until that call completes (it's value isn't directly exposed on `api-ch`). Depending on the success of `:get builds-url` it will put a failure message on the error channel or put a `:recent-builds` message in the `api-ch` itself, and then make two parallel server calls for the project's `settings` and `plan`. All of the api calls will put messages on `api-ch` when they return, and there's no ordering enforced here other than the sequencing of the requests themselves.
    `post-navigated-to! :dashboard` may call `api/get-projects`, and always calls `ajax/managed-ajax :get builds-url)`. In a goroutine, it blocks until that call completes (it doesn't put the response into `api-ch` the way `api` functions do). Depending on the success of `:get builds-url` it will put a failure message on the error channel or put a `:recent-builds` message in the `api-ch`, and then make two parallel server calls for the project's `settings` and `plan`. All of the api calls will put messages on `api-ch` when they return. There's no ordering enforced here for when responses are put on the channel, the sequencing is only for the order of making the requests.

    As messages are put in the `api-ch`, they are picked up in `core` by `api-handler`, which works similarly to the other handlers. It dispatches to a multimethod in the `api` controller to transition the state atom. This transition is straightforward for `projects` (https://github.com/circleci/frontend/blob/master/src-cljs/frontend/controllers/api.cljs#L98). The state transition for `recent-builds` (https://github.com/circleci/frontend/blob/master/src-cljs/frontend/controllers/api.cljs#L107) is a bit more involved, and here there's a bit more logic that appears to be guarding against race conditions with updating the state atom.
    As response messages are put in the `api-ch`, they are picked up in `core` by `api-handler`, which works similarly to the other handlers. It dispatches to a multimethod in the `api` controller to transition the state atom. This transition is straightforward for `projects` (https://github.com/circleci/frontend/blob/master/src-cljs/frontend/controllers/api.cljs#L98). The state transition for `recent-builds` (https://github.com/circleci/frontend/blob/master/src-cljs/frontend/controllers/api.cljs#L107) is a bit more involved, and here there's logic that appears to be guarding against race conditions between updating the state atom with the response, and the state atom having changed during the time it took to service the request.

    For `project-settings` (https://github.com/circleci/frontend/blob/master/src-cljs/frontend/controllers/api.cljs#L239) and `project-plan` (https://github.com/circleci/frontend/blob/master/src-cljs/frontend/controllers/api.cljs#L246), it looks like there's some other validation that these transitions make to check whether they make sense with other bits of the state atom (e.g., what the current project is), and no-op if they don't make sense. This is because the result of these server calls are being merged into the part of the state atom with data about the project.

    If I read this code correctly, there's a race condition here, but it's a bit tricky so I'm not quite sure and would love a check on my understanding. :) There's an optimization in `post-navigated-to! :dashboard` to not perform a `get-projects` request if the project already exists in the state atom. But on first page load the controller will make this server call. Let's assume it takes 15 seconds. The call to `:get builds-url` is made next and a goroutine in the controller blocks until it returns. Let's assume that one's fast and takes only 1 second. After that request completes, the state is updated and `:recent-builds` is set in the state atom. The `navigation` controller also makes requests for the project `settings` and `plan`. If those requests complete before the `projects` request completes, the guard in the `api` controller will prevent the state atom from being updated with the results of those calls. Then, after 12 seconds, the `projects` call completes and it updates the state atom. If the project's plan was a trial plan, this would result in the trial message never being show in the UI.
    If I read this code correctly, there's a race condition here, but it's a bit tricky so I'm not quite sure and would love a check on my understanding. :) There's an optimization in `post-navigated-to! :dashboard` to not perform a `get-projects` request if the project already exists in the state atom. But on first page load the controller will make this server call. Let's assume it takes 15 seconds. The call to `:get builds-url` is made next and a goroutine in the controller blocks until it returns. Let's assume that request is fast and takes only 1 second. After that request completes, the state is updated and `:recent-builds` is set in the state atom. The `navigation` controller also makes requests for the project `settings` and `plan`. If those requests complete before the original `projects` request completes, the guard in the `api` controller will prevent the state atom from being updated with the results of the `settings` and `plan` calls. Then, after 12 seconds, the `projects` call completes and it updates the state atom. If the project's plan was a trial plan, this would result in the trial message never being show in the UI.

    (Slight aside: If I read things correctly there's no way to cancel the server calls once `post-navigated-to! :dashboard` has been called, so it looks to me like it's possible to run into other race conditions here without guarding all these state transition functions. It'd be nice to split this guard out from the transition itself. Or to move the semantic validation of state transitions to somewhere central, or to limit the transitions that can be performed by restricting access to functions that can change the state map, rather than just `assoc`ing. Perhaps these guards handle all of those cases, but I'd be wary of race conditions hiding in here. I'm curious to check my understanding of this. Also would love to get some perspective from Clojure folk on the tradeoffs of using plain maps versus something that restricts state transitions to being semantically correct, and also the match of global state with pending server calls that are implicit state with no clear owner).
    > (Slight aside: If I read things correctly there's no way to cancel the server calls once `post-navigated-to! :dashboard` has been called, so it looks to me like it's possible to run into other race conditions here without guarding all these state transition functions. It'd be nice to split this guard out from the transition itself. Or to move the semantic validation of state transitions to somewhere central, or to limit the transitions that can be performed by restricting access to functions that can change the state map, rather than just `assoc`ing. Perhaps the guards written here handle all of those cases, but this is hard for me to see, so I'd be wary of race conditions hiding in here. I'm curious to check my understanding of this. Also would love to get some perspective from Clojure folk on the tradeoffs of using plain maps versus something that restricts state transitions to being semantically correct, and also the match of global state with pending server calls that are implicit state with no clear owner. But I'm getting ahead of myself :)).

    (Another aside: It's interesting to see that in this case the more complex setup is done as part of the `navigation` controller. There's some complexity in other `api` controller methods, like `post-api-event! [:build :success]` (https://github.com/circleci/frontend/blob/master/src-cljs/frontend/controllers/api.cljs#L143), so I'm curious to learn more about the rationale for deciding where to put the complexity, although I didn't look at this further yet).
    > (Another aside: It's interesting to see that in this case the more complex setup is done as part of the `navigation` controller. There's some complexity in other `api` controller methods, like `post-api-event! [:build :success]` (https://github.com/circleci/frontend/blob/master/src-cljs/frontend/controllers/api.cljs#L143), so I'm curious to learn more about the rationale for deciding where to put the complexity, although I didn't look at this further yet).
    So, as these API results come in, they'll update the state atom, and a view render will be triggered to update accordingly. But, backing up a bit, the first view change that would occur was from the first swap of `navigation-point` done in `defmethod navigated-to :default`. This would lead to the `app` component mounting the `dashboard` component as the `dominant-component` (https://github.com/circleci/frontend/blob/master/src-cljs/frontend/components/app.cljs#L130).

  27. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 8 additions and 10 deletions.
    18 changes: 8 additions & 10 deletions walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -16,11 +16,11 @@ Hello!

    This is a walkthough of what happens on startup in [https://github.com/circleci/frontend], looking at this SHA in December 2014: [https://github.com/circleci/frontend/commit/273558250040ce197e44e683d5528381f36c4eea].

    The first part is a literal walkthrough, where I'm tracing how I read the code works. This is a way to check my understanding, since of course I don't have all of your context. And it's also a way to echo back to you how someone else might experience this code without that context. The second part is about questions, suggestions, and having a conversation where over time we can sync on our understanding of how well the design here fits the product or application.
    The first part is a literal walkthrough, where I'm tracing my reading of how the code works. This is a way to check my understanding, since of course I don't have all of your context. And it's also a way to echo back to you how someone else might experience this code without that context. The second part is about asking questions, making suggestions, and having a conversation that, over time, will help us sync on our understanding of how well the design here fits the product or application.

    Doing this with colleagues has worked well for me in the past as a way to do in-depth code reviews about more abstract design and architectural questions. The goal of the the first part is to clarify we're all looking at the same thing, which creates the space and shared understanding to do a good critique in the second part where we both learn things. :)
    Doing this with colleagues has worked well for me as a way to do in-depth code reviews about more abstract design and architectural questions. The goal of the the first part is to clarify we're all looking at the same thing, which creates the space and shared understanding to do a good critique in the second part where we both learn things. :)

    One last note. My goal here is to learn more about these programming constructs and share some of the learnings I've had reading through this code. I'm sharing this with you because reading your work has already been enormously valuable to me, enough so that I believe it's worth it to work towards a more in-depth understanding of how you think about engineering.
    One last note, since we don't know each other. :) My goal here is to learn more about these programming constructs and share some of the learnings I've had reading through this code. I'm sharing this with you because reading your work has already been enormously valuable to me, enough so that I believe it's worth it to work towards a more in-depth understanding of how you think about engineering.

    So cheers! I'd welcome any thoughts or feedback you'd be willing to share back. :)

    @@ -31,21 +31,19 @@ Let's go!

    Ok, the entrypoing is `core/setup!`, called from CoffeeScript.

    It creates a new `history-imp`, which will listen for browser history events and will call `sec/dispatch!` as part of starting up. I'm assuming there's an event loop tick before this gets called, since otherwise it looks like `define-routes!` won't have been called yet.
    After defining an initial `state`, it creates a new `history-imp`, which will listen for browser history events and will call `sec/dispatch!` as part of starting up. I'm assuming there's an event loop tick before this gets called, since otherwise it looks like `main` and `define-routes!` won't have been called yet.

    The code in `core` sets everything up.
    In `main`, `app/app` gets mounted into the DOM.

    `app/app` gets mounted into the DOM.

    `:navigation-point` isn't set on the app-state, so the rendering code shows an empty div.
    `:navigation-point` isn't set on the app-state, so the `app` rendering code shows an empty div.

    The event loop yields, the route change is picked up by `routes` and let's assume it picks up the dashboard route. It puts an event in `nav-ch` with the `navigation-point` and `args` map pulled out of the URL.

    In `core`, `nav-handler` picks this up and calls two multi-methods in the `navigation` controller, one to update the state atom and another to perform other effects (like making API calls).

    In `defmethod navigated-to :dashboard`, (https://github.com/circleci/frontend/blob/master/src-cljs/frontend/controllers/navigation.cljs#L79), it takes the current state and returns an updated state (which the `nav-handler` uses to do a `swap!`). Here you can see the complexity of performing state transitions when everything is in a global atom, but also the benefits of this being dealt with explicitly (e.g., testable, clear seam for debugging).

    (As an aside, the comment at the top of the navigation controller about needing some other abstraction here really rings true. My instinct is that it's around restricting the semantics of state transitions though, rather than using middleware to help with this).
    > (As an aside, the comment at the top of the navigation controller about needing some other abstraction here really rings true. My instinct is that it'd be more important to restrict the semantics of state transitions, rather than building machinery to automatically do parts of the transition for each handler. I actually think the way you've composed this with the `->` and `assoc` is very clear).
    After swapping to the new state, the `nav-handler` calls `post-navigated-to! :dashboard` (https://github.com/circleci/frontend/blob/master/src-cljs/frontend/controllers/navigation.cljs#L91). This does some updates immediately, like `set-page-title!` and recording analytics. And most importantly, it performs several calls to the server and schedules a sequence of them.

    @@ -93,4 +91,4 @@ The other parts of UI programming that I think is truly difficult are the true d

    Of course, I'm writing you because I admire your work tremendously, and would really value any feedback or thoughts you had the time to share. Keep doing (and sharing) your awesome work! :)

    -Keviny
    -Kevin
  28. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 4 additions and 4 deletions.
    8 changes: 4 additions & 4 deletions walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -2,9 +2,9 @@ Brendon,

    My name's Kevin Robinson, and I'm super excited about your work!

    I read through the CircleCI frontend repo in some depth, and wrote up some notes and thoughts on it. I'm hoping you'll be open to reading it through and sharing your thoughts. Even if not, your work has been tremendously valuable and I'm grateful you're sharing it in blog posts and as open source. :)
    I read through the CircleCI frontend repo in some depth, and wrote up some notes and thoughts on it. I'm hoping you'll be open to reading it through and sharing your thoughts. Even if not, your work has been tremendously valuable and I'm grateful you're sharing it in blog posts and as open source. :)

    Here's a gist:
    Here's a gist:

    -Kevin
    [email protected]
    @@ -91,6 +91,6 @@ I'm fully sold on the functional idea of `data structures first`, with pure stat

    The other parts of UI programming that I think is truly difficult are the true distributed systems problems of data consistency. But in practice, most applications don't actually need to solve this, or the investment isn't worth it. Simpler approximations are almost always sufficient. Like taking a snapshot of the server state and holding it as immutable for a period of seconds or minutes, predicting that the likelihook of it changing underneath the user is very low and the path to recovery when it does happen is very straightforward. Or embracing eventual consistency and decoupling, and allowing data to be updated in different views within seconds of each other rather than synchronously. Or by investing in improvements to the API server layer so that server response times are reliably within 100s of milliseconds and more extensive browser/device-side caching is not worth the complexity.

    Of course, I'm writing you all because I admire your work tremendously, and would really value any feedback or thoughts you had the time to share. Keep doing (and sharing) your awesome work! :)
    Of course, I'm writing you because I admire your work tremendously, and would really value any feedback or thoughts you had the time to share. Keep doing (and sharing) your awesome work! :)

    -Kevin
    -Keviny
  29. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 30 additions and 6 deletions.
    36 changes: 30 additions & 6 deletions walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -1,21 +1,45 @@
    Brendon,

    My name's Kevin Robinson, and I'm super excited about your work!

    I read through the CircleCI frontend repo in some depth, and wrote up some notes and thoughts on it. I'm hoping you'll be open to reading it through and sharing your thoughts. Even if not, your work has been tremendously valuable and I'm grateful you're sharing it in blog posts and as open source. :)

    Here's a gist:

    -Kevin
    [email protected]
    @krob



    Hello!

    This is a walkthough of what happens on startup in [https://github.com/circleci/frontend], looking at this SHA in December 2014: https://github.com/circleci/frontend/commit/273558250040ce197e44e683d5528381f36c4eea. The first part is a literal walkthrough, where I'm tracing how I read the code as a way to both check my understanding and echo back to you how I see what's happening here, since of course I don't have all your context. :) This has worked well with colleagues in the past as a way to do in-depth code reviews about design and architectural questions, where the first section is trying to clarify we're all looking at the same thing, and the second is trying to evaluate or critique the strengths and weaknesses of what we see. Please remember I'm doing my best to do this in a constructive fashion, and that I took the time to do this and share it with you because reading your work has been enormously valuable to me and I would welcome any feedback you have. :)
    This is a walkthough of what happens on startup in [https://github.com/circleci/frontend], looking at this SHA in December 2014: [https://github.com/circleci/frontend/commit/273558250040ce197e44e683d5528381f36c4eea].

    The first part is a literal walkthrough, where I'm tracing how I read the code works. This is a way to check my understanding, since of course I don't have all of your context. And it's also a way to echo back to you how someone else might experience this code without that context. The second part is about questions, suggestions, and having a conversation where over time we can sync on our understanding of how well the design here fits the product or application.

    Doing this with colleagues has worked well for me in the past as a way to do in-depth code reviews about more abstract design and architectural questions. The goal of the the first part is to clarify we're all looking at the same thing, which creates the space and shared understanding to do a good critique in the second part where we both learn things. :)

    One last note. My goal here is to learn more about these programming constructs and share some of the learnings I've had reading through this code. I'm sharing this with you because reading your work has already been enormously valuable to me, enough so that I believe it's worth it to work towards a more in-depth understanding of how you think about engineering.

    So cheers! I'd welcome any thoughts or feedback you'd be willing to share back. :)

    -Kevin

    Let's go!
    ---

    Entrypoint is core/setup!, called from CoffeeScript.
    Ok, the entrypoing is `core/setup!`, called from CoffeeScript.

    It creates a new `history-imp`, which will listen for browser history events and will call `sec/dispatch!` as part of starting up. I'm assuming there's an event loop tick before this gets called, since otherwise it looks like `define-routes!` won't have been called yet.

    The code in core sets everything up.
    The code in `core` sets everything up.

    app/app gets mounted into the DOM.
    `app/app` gets mounted into the DOM.

    :navigation-point isn't set on the app-state, so the rendering code shows an empty div.
    `:navigation-point` isn't set on the app-state, so the rendering code shows an empty div.

    Event loop yields, route change is picked up by `routes` and let's assume it picks up the dashboard route. It puts an event in `nav-ch` with the `navigation-point` and `args` map pulled out of the URL.
    The event loop yields, the route change is picked up by `routes` and let's assume it picks up the dashboard route. It puts an event in `nav-ch` with the `navigation-point` and `args` map pulled out of the URL.

    In `core`, `nav-handler` picks this up and calls two multi-methods in the `navigation` controller, one to update the state atom and another to perform other effects (like making API calls).

  30. kevinrobinson revised this gist Jan 2, 2015. 1 changed file with 7 additions and 0 deletions.
    7 changes: 7 additions & 0 deletions walkthrough.md
    Original file line number Diff line number Diff line change
    @@ -1,3 +1,10 @@
    Hello!

    This is a walkthough of what happens on startup in [https://github.com/circleci/frontend], looking at this SHA in December 2014: https://github.com/circleci/frontend/commit/273558250040ce197e44e683d5528381f36c4eea. The first part is a literal walkthrough, where I'm tracing how I read the code as a way to both check my understanding and echo back to you how I see what's happening here, since of course I don't have all your context. :) This has worked well with colleagues in the past as a way to do in-depth code reviews about design and architectural questions, where the first section is trying to clarify we're all looking at the same thing, and the second is trying to evaluate or critique the strengths and weaknesses of what we see. Please remember I'm doing my best to do this in a constructive fashion, and that I took the time to do this and share it with you because reading your work has been enormously valuable to me and I would welcome any feedback you have. :)

    Let's go!
    ---

    Entrypoint is core/setup!, called from CoffeeScript.

    It creates a new `history-imp`, which will listen for browser history events and will call `sec/dispatch!` as part of starting up. I'm assuming there's an event loop tick before this gets called, since otherwise it looks like `define-routes!` won't have been called yet.