state of ambermonkey paper

gistfile1.txt

Main Updates from Last week:

- First story change: Reducing the emphasis on AOT Baseline compilation,
  elevating the contribution of AOT ICs. Making the argument that a
  stuctured Inline Caching IR and separation from code and data enables the
  high coverage we see. AOT baseline compilation is kept around only as a
  contrast.

    - I'm still working out the arguments for section III. I will work on
      this tongiht to get your thoughts tomrorrow or later here.

    - I separated the security background into it's own section and included
      some more modern references. Will have to have your thoughts on these,
      especially the Nebula security issue. Section III will now focus on
      the SpiderMonkey background and emphasize CacheIR and it's design, how
      it makes AOT ammenible.

- Second story change: Wanted to mention PBL and Weval more, I think more
  credit is due here for Chris' work finding that a reasonably small IC
  corpus served the needs of PBL.

- Defining restricted execution into two categories. Namely "No runtime
  codegen" and "Trusted Initialization Only". Still working on the names
  here. Helps distinguish us from V8 JITless and lets us divulge no WASM
  interp, r.t trampolines, etc, upfront.

- Evaluations:
    
    1. Got rid of the "soft threshold" concept for accruing a corpus. Define
       tp6_train and tp6_test partitioned into [24:8]. Including all ICS
       from tp6_train actually gives JetStream 98% IC coverage (formerly
       below 80%)

    2. Unrolled Speedometer3 Speedup graph to show per-subbench
       speedups.Should we do another graph for JetStream?

    3. Measuring resident memory at intervals on Speedometer actually gives
       much nicer memory sharing statistics than AWSY + tp6. For some
       reason interp-only spawns double the JS runtimes on Speedeomter
       three. 

Questions:

- What _really_ is the key design principle of CacheIR that makes our
  AOT corpus so small in practice? I don't understand enough about other
  IC systems to contrast.

- Should we add an AOT image loading ablation? This would be timings for how
  long each part of the AOT loading takes, compared with Baseline compilation.

- Thoughts on the arguments / claims in _The Complexity of Optimizing JavaScript_ 
   new background on JIT Security.

Most remaining uncertainties:

- Memory results. We can't cleanly claim memory reduction. We seem to occupy
  some space inbetween interpretation and Baseline compilation in terms of
  memory consumption. AmberMonkey has no apples to apples comparrison with
  another tier. We can claim we _observe_ sharing, which is the stated goal
  of indirection.

- The V8 JITles comparisson would need to be qualified with "we have WASM
  codegen turn on". Still need to turn off Regex for the Speedometer3 perf
  results.

- The Indirection Overhead is overly pessimistic by using targeted
  microbench marks. I like the honesty of this. Maybe another column in the
  figure, with a separate Y-axis for stock Baseline against stock Baseline +
  AOT? (The configuration Chase used to find his original ~5% overhead)

- Can we call the optimizations "optimizations"? We need to measure thier
  isolated impact if so.

Stuff I'm still planning to do:

- Define, once and for all, all the execution configurations and conform all
  prose, figure labels, and even color schemes to these definitions.

- Some way to attribute AmberMonkey performance _per artifact_. Recent AI
  evaluations have took issue to this: Can't tell what is self-hosted, what
  is blinterp, what is IC.

I will update the typst link shortly. Here is repo's PDF. Section III prose is unsettled. https://github.com/JustinMeimar/ambermonkey/blob/main/paper/draft.pdf