We also implemented a JIT-based `rescale` for `ArrayFlow`, where the
same function as for `LazyArrayRangeFlow` passes through and can do an
arbitrary symbolic shift/rescale/order-preserving whatever.
To make this fast for `ArrayFlow`, we utilize a "test" variable `a`,
put it through the function and rescale/strip its units, then `lambdify`
it and broadcast the function onto `ArrayFlow.values`.
It's an immensely clean way of working.
The `lambdify` seems fast, interestingly, but ideally we would of course
also cache that somehow.
Some details remain:
- Fourier Transform index bounds / length are currently not presumed
known before actually computing them; it's unclear what's best here,
as the design cross-section of physical expectation, mathematical
correctness, and ease of implementation (especially trying to keep
actual data out of the `InfoFlow` hot-path). For now, the `0..\infty`
bounds **will probably break** the `Viz` node.
- We've reverted the notion that particular `sim_symbol`s must have a
unit pre-defined, which causes a little more complexity for nodes like
`TemporalShape`. This question is going to need resolving
- The whole `InfoFlow` object really seems to be at the heart of certain
lagginess when it comes to the math system. It, together with the
index representations, would benefit greatly from a principled
refactor.
- The `Viewer` node is broken for 3D preview; see #70.
Closes#59. Progress on #54.
We've added enough nodes to run simulations, and there is now only an
"air gap" remaining between the math nodes and the sim design nodes.
Of particular importance right now are:
- Finalizing the exporter w/Sim Data output.
- Finishing the plane wave, as well as other key source nodes (TFSF,
Gaussian Beam, Plane Wave stand out in particular).
- Reduce node, as we need to be able to reconstruct total flux
through a field monitor, as well as generally do statistics.
- Transform node, particularly w/pure-InfoFlow reindexing (we need to be
able to reindex frequency -> vacwl), fourier transform (we need to be
able to cast a time-domain field recording to frequency domain).
- Finish debugging the basic structures, in particular the Cylinder
primitive, but also the generic GeoNodes node, so we can build the
membrane properly.
- Ah yes, also, the grid definition. The default can sometimes act a
little suspiciously.
It now supports selecting the variant, and exporting a generated
LazyArrayRange of valid freqs/wls.
The UI was also revamped, with greatly more readable range values,
dynamic labels, link button pointing to the original data, etc. .
While more LOC, the code structure is also far more explicit and predictable to
maintain.
A very healthy amount of research on how to choose the Bloch vector was
performed.
It is encapsulated not only in the documentation, but also in the modes
available for how to derive one that fits a given simulation.
The theory of the Bloch boundaries can really bite you, and the hope is
that by focusing on such invalid-usage-prevention, a lot of time can be
saved in the sim design stages.
We now have a solid category (w/accompanying sockets) for defining
boundary conditions.
We also have a single-boundary-condition node for fully configuring the
all-important PML condition.
Some insights:
- PEC/PMC are so dead-simple that giving them their own nodes doesn't
even make sense.
- StablePML and PML are the same, just with differing layers. Chose
to require the user add layers to the PML node for the same effect.
- "Periodic" is a special case of "Bloch", so we only need "Bloch".
- "Absorber" vs "PML" is an important choice for the user, which we
must ensure shines through.
I'm of the belief that the correct abstractions are now actually
available, and that most-to-all of the required functionality actually
already exists within the code base.
The art is bringing it together!
BLField has gotten a huge facelift, to make it practical to wrangle
properties without the sharp edges.
- All the "special" UI-exposed property types can now be directly
constructed in a BLField marked with 'prop_ui'.
- The most appropriate internal representation will be chosen to
represent the attribute based on its type annotation, including sized
vector-like `bool`, `int`, `float` for `tuple[...]`.
- Static EnumProperties can now be derived from a special `StrEnum`, to
which a `to_name` and `to_icon` method is attached.
- Dynamic `EnumProperty` can now be used safely, with builtin
workarounds to the real-world reference-loss-crash (realized
in the Tidy3D Cloud Task node) and jankiness like empty enum.
- The update method is now fully managed, negating all bugs related to
improper update callback naming.
- Python-side getter caching is preserved for ui-exposed
properties, with the help of node/socket base class support for
passing a `Signal.InvalidateCache` to BLFields that are altered in the
UI.
The cost to all this niceness is rather low, and arguably, positive:
- Dynamic Enum/String searchers no longer "magically" invoke all the
time, since the values seen by Blender are cached by the BLField.
- To regenerate the searcher output, an `@on_value_changed` should be
made by the user to pass `Signal.ResetEnumItems` or
`Signal.ResetStrSearch` to the `BLField`.
- Since searching is no longer eager, there is no danger of
out-of-reference strings (which crash Blender from EnumProperty), but
also a greatly reduced performance problems associated with
the hot-loop regeneration of EnumProperty strings.
- The base classes are now involved with BLField invalidation, to ensure
that the getter caches are cleared up when the UI changes. For the
price of that small indirection (done cheaply with set lookup),
all attribute lookups are generally done in a single lookup, completely
avoiding Blender until needed.
- This does represent another increase in confidence wrt. the event
system's integrity, but so far, that has been a very productive
direction.
**NOTE**: The entire feature set of BLField is not well tested, and will
likely need adjustments as the codebase is converted to use them.
Enormously important changes to the data flow semantics and invalidation
rules. Especially significant is the way in which the node graph
produces a deeply composed function, compiles it to optimized machine
code with `jax`, and uses a seperately cached data flow to insert values
into the function from anywhere along the node graph without recompiling
the function.
A critical portion of the math system, namely the unit-aware dimensional
representation, is also finished. The `Data` node socket type now
dynamically reports the dimensional properties of the object flowing
through it, courtesy the use of a seperate data flow for information.
This allows for very high-peformance unit-aware nearest-value indexing built on binary
search.
Also, dependency management is completely ironed out. The `pip install`
process now runs concurrently, and the installation log is parsed in the
background to update a progress bar. This is the foundational work for a
similar concurrent process wrt. Tidy3D progress reporting.
The serialization routines are fast and effective.
Overall, the node graph feels snappy, and everything updates smoothly.
Logging on the action chain suggests that there aren't extraneous calls,
and that existing calls (ex. no-op previews) are fast.
There will likely be edge cases, and we'll see how it scales - but
for now, let's go with it!
This especially involved fixing the invalidation logic in
`trigger_action`.
It should now be far more accurate, concise, and performant.
The invalidation check ought still be optimized.
The reason this isn't trivial is because of the loose sockets:
To use our new `@keyed_cache` on a function like `_should_recompute_output_socket`, the loose
socket would also need to do an appropriate invalidation.
Such caching without accounting for invalidation on loose-socket change
would be incorrect.
For now, it seems as though performance is quite good, although it is
unknown whether this will scale to large graphs.
We've also left `kind`-specific invalidation alone for now (maybe
forever).
The crash: When a linked loose socket was deleted, the link remained in the
NodeLinkCache, and caused a crash when trying to ask the already-deleted
socket for removal consent. We fixed this by reporting all socket
removals to the node tree, so that links could be correctly removed
independently of link-change calculation.
The bug: The collection getter was cached improperly; Blender ID types
can't just be saved like that. We need to search every time. Performance
seems unaffected at first glance.
This is essential for:
- Representing ranges as bounds
- Arbitrary symbolic/numeric representation of spectral distributions
- Parametric representation and JIT of critical-path procedures.
Unfortunately this broke a lot of nodes in small ways.
Next step is to finish the low-hanging fruit nodes + fix the ones we
have.
It's only good for dispersive media; specifically, a text file with
three floats per line: 'wl n k'.
A custom script was used to convert Maxim's data.
It's very fast, and has a ton of options.
Only the most important are exposed in the node for now.
A bug in MPL plotting aspect ratio declaration on MPL axis object was
fixed by manually running `set_aspect('auto')` after the fact.
It shouldn't do anything, and it doesn't, other than fix the bug :)
Also brought the plotting function of the viewer to parity with
the 3D preview, so the "Auto Plot" button works as expected.
