Commit Graph

95 Commits (c9936b89423a68222ec44488be350f741b977440)

Author SHA1 Message Date
Sofus Albert Høgsbro Rose c9936b8942
feat: big refactor to fight fundamental crashes
Some rather foundational things were fundamentally broken, especially related to the initialization procedures of fields / cached properties.

- We completely revamped `bl_cache`, fixing many to-be-discovered bugs.
- We completely streamlined `BLField` property logic into reusable
  `bl_cache.BLProp` and `bl_cache.BLPropType`.
- We implemented `BLInstance` superclass to handle ex. deterministic
  persistance of dynamic enum items, and other nuanced common
  functionality that was being duplicated raw.
- We implemented inter `cached_bl_property` / `BLField` dependency
  logic, including the ability to invalidate dynamic enums without
  @on_value_changed logic. This **greatly** simplifies a vast quantity
  of nodes that were traditionally very difficult to get working due to
  the sharp edges imposed by needing manual invalidation logic.
- We gave `ExprSocket` a significant usability upgrade, including
  thorough parsing logic in the `SocketDef`.

It's not that existing nodes are as such broken, but their existing bugs
are now going to cause problems a lot faster.
Which is a good thing.

BREAKING CHANGE: Closes #13. Closes #16. Big work on #64. Work on #37.
2024-05-15 12:37:38 +02:00
Sofus Albert Høgsbro Rose 929fb2dae9
feat: implemented `GaussianBeam` source
This node is good for approximating a simple laser.
2024-05-07 12:39:17 +02:00
Sofus Albert Høgsbro Rose 9f8ff33e4f
feat: implement plane wave node w/viz
the visualization of Tidy3D's spherical coordinates was exceptionally harsh. See #63 for more information.

Closes #63.
2024-05-06 22:11:05 +02:00
Sofus Albert Høgsbro Rose 0fbd3752b3
feat: real-time videos of simulated EM fields
We did it! We have a `Scene` node which outputs the current frame/time
mapped to a friendlier unit (like `ps`) and takes an integer FPS.

A `frame_post` handler in Blender makes sure to run `DataChanged` on any
`Scene` nodes in any active node trees (a notion we added with this
commit), which causes a `@property` to effectively masquerade as a
"real" property. Thus, all the caching/other stuff like that has its
semantics preserved, so long as the invalidated `@property` stays.

We can probably optimize the handler - it linearly scans all nodes in
all trees. A bit suboptimal. We can also think more about more outputs
and such. Still, a fine start, and very cool with videos of EM fields
doing their thing! (Of course, one can plug this into anything, not just
the time-fixing part of an Expr chain).

Closes #17.
2024-05-05 17:57:07 +02:00
Sofus Albert Høgsbro Rose 084ae632f1
fix: fixed bugs preventing end-to-end sim
Several bugs/gotchas/papercuts have been fixed, which together made it really hard

Closes #47 #48 #49 #50 #51 #52.
2024-05-05 14:27:02 +02:00
Sofus Albert Høgsbro Rose 999d601c49
fix: fixed web runner import 2024-05-05 10:00:45 +02:00
Sofus Albert Høgsbro Rose b0db0ca072
fix: fix import and remove some TODOs
We're slowly migrating all the random sprinkled TODOs into actual, trackable issues, so we actually have a change of keeping an overview.
2024-05-05 09:51:17 +02:00
Sofus Albert Høgsbro Rose c5f438bb87
docs: added license headers to all files
It's recommended to add license headers to all files in an AGPL project, in case a file is viewed outside the context of its main repository. We're using a `pre-commit` tool to manage this, to make sure it's consistently applied to all our Python files.
2024-05-04 22:29:40 +02:00
Sofus Albert Høgsbro Rose 691ff4a393
bump: version 0.0.1 → 0.1.0 2024-05-04 20:09:16 +02:00
Sofus Albert Høgsbro Rose a7e3c17c86
refactor: applied tooling for predictable lint/fmt/commits
Applied `rye lint --fix`, `rye fmt`, and commitizen checking to better control the project development.
2024-05-04 20:08:33 +02:00
Sofus Albert Høgsbro Rose ea8e4104ff
fix: Run `active_kind` updator after socket init.
Without propagation, of course; this ensures that the display shape is correctly set.

Closes #2.
2024-05-04 16:03:59 +02:00
Sofus Albert Høgsbro Rose 3c00530524
fix: Unit conversion of LazyValueRange.
The unit conversion was indeed botched, with a typo causing the start of
the range to be converted as if it were the end of the range.

Closes #3.
2024-05-04 15:51:40 +02:00
Sofus Albert Høgsbro Rose 3a53e4ce46
refactor: Factored out flow_kinds.py for clarity. 2024-05-04 11:25:11 +02:00
Sofus Albert Høgsbro Rose b221f9ae2b
feat: E2E simulation design and analysis.
This is it!
This is the milestone.
We can now make, run, and analyze simulations in one big chain.

Jank remains:
- Dynamic enums still need caching, lest the user think to restart
  Blender while the math nodes are riding high on `FlowPending`.
- GN remains untested, and so forth.
- Still no plane wave node. Easy to jank together, though.
- Still no reindexing in the Transform math, so only frequencies for
  now.
- Active kinds still don't update shape, we still need an explicit
  (postinit?) directive to do that.
- No colors for expr sockets :(

But we have a beautifully solid foundation to work on.
The new abstractive tools for defining event-driven actions via nodes
have had very few showstoppers, and are incredibly nice to work with.
There are sharp edges, of course, but generally they only matter where
the problem was so very difficult to begin with.

We'll start doing physics immediately, and fixing bugs / implementing
more nodes as we go.
2024-05-03 11:47:51 +02:00
Sofus Albert Høgsbro Rose 695eedca98
feat: Feature-parity with past.
We've added enough nodes to run simulations, and there is now only an
"air gap" remaining between the math nodes and the sim design nodes.

Of particular importance right now are:
- Finalizing the exporter w/Sim Data output.
- Finishing the plane wave, as well as other key source nodes (TFSF,
  Gaussian Beam, Plane Wave stand out in particular).
- Reduce node, as we need to be able to reconstruct total flux
  through a field monitor, as well as generally do statistics.
- Transform node, particularly w/pure-InfoFlow reindexing (we need to be
  able to reindex frequency -> vacwl), fourier transform (we need to be
  able to cast a time-domain field recording to frequency domain).
- Finish debugging the basic structures, in particular the Cylinder
  primitive, but also the generic GeoNodes node, so we can build the
  membrane properly.
- Ah yes, also, the grid definition. The default can sometimes act a
  little suspiciously.
2024-05-03 00:25:34 +02:00
Sofus Albert Høgsbro Rose 9df0d20c68
feat: Finished Gaussian Pulse node.
Also fixed several bugs along the way.
Full speed aheaad on the sources!
2024-05-02 20:59:30 +02:00
Sofus Albert Høgsbro Rose 14b98d219e
feat: Finished Library Medium node.
It now supports selecting the variant, and exporting a generated
LazyArrayRange of valid freqs/wls.

The UI was also revamped, with greatly more readable range values,
dynamic labels, link button pointing to the original data, etc. .

While more LOC, the code structure is also far more explicit and predictable to
maintain.
2024-05-02 14:24:23 +02:00
Sofus Albert Høgsbro Rose 339ee0226d
feat: Added the Bloch boundary condition.
A very healthy amount of research on how to choose the Bloch vector was
performed.
It is encapsulated not only in the documentation, but also in the modes
available for how to derive one that fits a given simulation.

The theory of the Bloch boundaries can really bite you, and the hope is
that by focusing on such invalid-usage-prevention, a lot of time can be
saved in the sim design stages.
2024-05-02 11:12:33 +02:00
Sofus Albert Høgsbro Rose 2f42c9d91b
feat: Added adiabatic absorber.
It's useful for scenarios where we need to intersect geometry with the
simulation boundary.
Generally, not preferred, which we make clear in the docs :)

Also fixed a bug in `events.py` that was misjudging `FlowInitializing`,
and causing `BoundConds` to fail.
Weird.
Anyway, fixed!
2024-05-01 16:38:11 +02:00
Sofus Albert Høgsbro Rose f60b736584
feat: Added BoundConds Node & Fancy PML Node
We now have a solid category (w/accompanying sockets) for defining
boundary conditions.
We also have a single-boundary-condition node for fully configuring the
all-important PML condition.

Some insights:
- PEC/PMC are so dead-simple that giving them their own nodes doesn't
  even make sense.
- StablePML and PML are the same, just with differing layers. Chose
  to require the user add layers to the PML node for the same effect.
- "Periodic" is a special case of "Bloch", so we only need "Bloch".
- "Absorber" vs "PML" is an important choice for the user, which we
  must ensure shines through.
2024-05-01 16:06:23 +02:00
Sofus Albert Høgsbro Rose 7263d585e5
fix: Inching closer.
I'm of the belief that the correct abstractions are now actually
available, and that most-to-all of the required functionality actually
already exists within the code base.
The art is bringing it together!
2024-05-01 13:54:16 +02:00
Sofus Albert Høgsbro Rose e330b9a451
refactor: Huge simplifications from ExprSocket 2024-04-30 18:42:46 +02:00
Sofus Albert Høgsbro Rose 80d7b21c34
refactor: Big breakthrough on Expr socket (non working) 2024-04-28 18:02:54 +02:00
Sofus Albert Høgsbro Rose d8170a67e4
plan: Working TODO ready 2024-04-27 20:05:57 +02:00
Sofus Albert Høgsbro Rose 8012415278
plan: Revamped TODO w/final planning 2024-04-27 19:49:46 +02:00
Sofus Albert Høgsbro Rose 0599eed2b2
enhance: Streamlined GN asset system 2024-04-27 15:55:15 +02:00
Sofus Albert Høgsbro Rose 4e1eb19a88
feat: Use `canvas.draw()` for plotting.
The performance difference isn't as clear cut as hoped.
However, the plotting procedure is enormously more straightforward, and
performance is more predictable.
So it's worth it.

We're managing to perfectly reuse figure/canvas/axis, but still hovering at around 70-80ms.
Mind you, the tested machine is an older laptop.
Still, things feel interactive enough, especially together with the
other modifications.

To really amp it up, we can look into blitting. It requires alterations
to the plotting methodology, but it offers a cached approach to drawing
only altered pixels (the bottleneck with `canvas.draw()` is that it
needs to render all the pixels, every time).

We can also try to lower the resolution if it's too slow.
2024-04-27 03:09:47 +02:00
Sofus Albert Høgsbro Rose c63dda2224
fix: Major streamlining of plot workflow.
Various small adjustments with a big total impact:
- Toggling 3D preview no longer propagates a DataChanged, which prevents
  chronic `bl_select()` invocation that greatly slows down switching
  between images.
- Viewer now uses a replot() context manager to hide any active plots
  whenever no plots were generated, so that turning off plots /
  previewing a node (chain) without plots properly turns off the image,
  instead of letting the image hang around.
- `self.managed_objs` is now properly invalidated, instead of trying to
  set the `name` attribute of in-memory objects, so that persistance
  keeps up with `sim_node_name` changes, so that all the ex. 'Viz' nodes
  don't all try to hog a single 'Viz' image name.
- A pre-save handler was added, which ensures all images are packed into
  the .blend, to ensure that the images will pop up on the next file
  load.
- A fake user is now assigned to all new images, to nail down the idea
  that `ManagedBLImage` is the owner.
- `name` setter of `ManagedBLImage` was unreasonably bugged (it's
  actually incredible that it worked) - it has been fixed, as well as
  other changes applied to the class as a whole (including @classmethod
  on the UI-area/space getters and minor None-sanitizing).
2024-04-27 01:52:04 +02:00
Sofus Albert Høgsbro Rose fc0d7afa4d
ui: Added dynamic labels to map/filter 2024-04-26 22:18:49 +02:00
Sofus Albert Høgsbro Rose 82e8e3f348
ui: Hide column selectors with columns 2024-04-26 19:25:42 +02:00
Sofus Albert Høgsbro Rose b2a7eefb45
feat: Implemented operate math node. 2024-04-26 17:22:55 +02:00
Sofus Albert Høgsbro Rose 7d944a704e
ui: Data socket UI is now spiffy. 2024-04-25 09:56:21 +02:00
Sofus Albert Høgsbro Rose badadfbfc2
fix: Extract fixes incl. draw, array-copy on export. 2024-04-24 19:06:15 +02:00
Sofus Albert Høgsbro Rose 785995117e
fix: Invalidate cache of removed input sockets.
This was a nasty (interesting?) one - usually, input sockets are not attempted used
after the socket no longer exists.
Various checks in ex. `events` tend to help that process along.

Unfortunately (fortunately?), `Extract` uses a `_compute_input` query with
`optional=True`, which results in a direct attempt to hit the cache
without any other checks.
Because old input socket caches were never deleted, it would
**continue to get cached data from sockets that no longer exist**.

While on the surface this could be considered a case of "the
private method (`_compute_input`) is private for a reason", or
alternatively, "don't hijack the graph flow", I'm more convinced that
the usage is actually quite clean, being read-only and generally
well-conceived. It's reasonable to presume that asking for a thing that
doesn't exist won't produce output!

Moreover, I wouldn't be surprised if several other mysterious bugs were
caused by this. Not to mention the memory leak of endless caching! (Well,
until the node is deleted). It's a good things we noticed!
2024-04-24 18:41:06 +02:00
Sofus Albert Høgsbro Rose c82862dde9
fix: Implement explicit no-flow w/FlowSignal 2024-04-24 18:36:29 +02:00
Sofus Albert Høgsbro Rose a3defd3c1c
feat: Complete matplotlib plotting system.
The Viz node now detects the shape of the data, and presents compatible
plot options.
Not all are implemented, but a few quite important ones are.

Additionally, a number of dataflow-related bugs were investigated and
fixed. A few were truly damaging, but many simply resulted in gross
inefficiencies - we must be careful declaring BLFields that are updated
in hot loops!

Moreover, it is exceptionally easy to add more as needed, as we analyze
more and more sims.
The only limit is `matplotlib`, which is... well, yeah.

Due to the BLField work, the dynamicness of the Viz node is quite
under control, so there will not be any critical issues there.

The plotting lags (70ms total in the hot loop), but that's actually
entirely fixeable.
It's also entirely the `managed_bl_image`'s fault.
Fixing these inefficiencies will also make Tidy3D's builtin plots
near-realtime, incidentally.

We profiled the following currently:
- 25ms: Creating `fig = plt.subplots`. We can reuse fig per-managed
  image.
- 43ms: The BytesIO roundtrip, including `savefig`. We can instead use
  the Agg backend, `fig.canvas.draw()`, and a `np.frombuffer` to both
  plot directly to the memory location,
- ~3ms: Actual plotting functions in `image_ops`. They are seriously fast.
- ~0ms: Blitting pixels to the Blender image - this was optimized in
  4.1, and it shows; the time to copy the data over is essentially nothing.
2024-04-23 19:27:45 +02:00
Sofus Albert Høgsbro Rose e7d3ecf48e
fix: Crashes on enum changes 2024-04-23 11:51:24 +02:00
Sofus Albert Høgsbro Rose 7fa6e3a3ec
fix: Crashiness of EnumProperty 2024-04-23 10:45:44 +02:00
Sofus Albert Høgsbro Rose f09b58e0e7
fix: BLFields in FilterMath, bug fixes. 2024-04-23 09:30:26 +02:00
Sofus Albert Høgsbro Rose 44d61b5639
feat: Safe, practical BLField.
BLField has gotten a huge facelift, to make it practical to wrangle
properties without the sharp edges.
- All the "special" UI-exposed property types can now be directly
  constructed in a BLField marked with 'prop_ui'.
- The most appropriate internal representation will be chosen to
  represent the attribute based on its type annotation, including sized
  vector-like `bool`, `int`, `float` for `tuple[...]`.
- Static EnumProperties can now be derived from a special `StrEnum`, to
  which a `to_name` and `to_icon` method is attached.
- Dynamic `EnumProperty` can now be used safely, with builtin
  workarounds to the real-world reference-loss-crash (realized
  in the Tidy3D Cloud Task node) and jankiness like empty enum.
- The update method is now fully managed, negating all bugs related to
  improper update callback naming.
- Python-side getter caching is preserved for ui-exposed
  properties, with the help of node/socket base class support for
  passing a `Signal.InvalidateCache` to BLFields that are altered in the
  UI.

The cost to all this niceness is rather low, and arguably, positive:
- Dynamic Enum/String searchers no longer "magically" invoke all the
  time, since the values seen by Blender are cached by the BLField.
- To regenerate the searcher output, an `@on_value_changed` should be
  made by the user to pass `Signal.ResetEnumItems` or
  `Signal.ResetStrSearch` to the `BLField`.
- Since searching is no longer eager, there is no danger of
  out-of-reference strings (which crash Blender from EnumProperty), but
  also a greatly reduced performance problems associated with
  the hot-loop regeneration of EnumProperty strings.
- The base classes are now involved with BLField invalidation, to ensure
  that the getter caches are cleared up when the UI changes. For the
  price of that small indirection (done cheaply with set lookup),
  all attribute lookups are generally done in a single lookup, completely
  avoiding Blender until needed.
- This does represent another increase in confidence wrt. the event
  system's integrity, but so far, that has been a very productive
  direction.

**NOTE**: The entire feature set of BLField is not well tested, and will
likely need adjustments as the codebase is converted to use them.
2024-04-23 07:55:54 +02:00
Sofus Albert Høgsbro Rose b4d6eae036
feat: High-performance math system and depsflow.
Enormously important changes to the data flow semantics and invalidation
rules. Especially significant is the way in which the node graph
produces a deeply composed function, compiles it to optimized machine
code with `jax`, and uses a seperately cached data flow to insert values
into the function from anywhere along the node graph without recompiling
the function.

A critical portion of the math system, namely the unit-aware dimensional
representation, is also finished. The `Data` node socket type now
dynamically reports the dimensional properties of the object flowing
through it, courtesy the use of a seperate data flow for information.
This allows for very high-peformance unit-aware nearest-value indexing built on binary
search.

Also, dependency management is completely ironed out. The `pip install`
process now runs concurrently, and the installation log is parsed in the
background to update a progress bar. This is the foundational work for a
similar concurrent process wrt. Tidy3D progress reporting.
2024-04-22 15:26:15 +02:00
Sofus Albert Høgsbro Rose 9960cd3480
refactor: Big changes to data flow and deps loading 2024-04-19 16:53:24 +02:00
Sofus Albert Høgsbro Rose ff5d71aeff
refactor: More changes to docs/layout 2024-04-18 08:42:53 +02:00
Sofus Albert Høgsbro Rose 8dece384ad
refactor: Moved contracts + fixes 2024-04-17 18:36:11 +02:00
Sofus Albert Høgsbro Rose 6a41683576
refactor: Use cleaner relative import for top-level `utils` 2024-04-17 18:16:14 +02:00
Sofus Albert Høgsbro Rose c6e00dcd7b
refactor: Fixes and movement. 2024-04-17 18:14:14 +02:00
Sofus Albert Høgsbro Rose 29cee2e7a2
fix: Some renamed FlowKinds (not all) 2024-04-17 16:17:13 +02:00
Sofus Albert Høgsbro Rose 76d15b0c92
refactor: Renamed DataFlowKind to FlowKind 2024-04-17 16:06:24 +02:00
Sofus Albert Høgsbro Rose dfeb65feec
feat: Math nodes (non-working) 2024-04-17 16:03:15 +02:00
Sofus Albert Høgsbro Rose 568fc449e8
fix: Caching now (seems to) work robustly.
The serialization routines are fast and effective.
Overall, the node graph feels snappy, and everything updates smoothly.
Logging on the action chain suggests that there aren't extraneous calls,
and that existing calls (ex. no-op previews) are fast.

There will likely be edge cases, and we'll see how it scales - but
for now, let's go with it!
2024-04-15 19:15:20 +02:00