Commit Graph

138 Commits (uv-refactor)

Author SHA1 Message Date
Sofus Albert Høgsbro Rose b921c3b5cd
refactor: Split assets to node-tree/general variants 2024-10-03 07:29:32 +02:00
Sofus Albert Høgsbro Rose 0d5c9b78f7
fix: Wrong string type a function annotation 2024-10-03 07:14:22 +02:00
Sofus Albert Høgsbro Rose fdcd52e937
fix: Automated ruff fixes. 2024-10-02 13:08:40 +02:00
Sofus Albert Høgsbro Rose 06ad0d62a5
fix: Fix tests. 2024-10-02 12:58:47 +02:00
Sofus Albert Høgsbro Rose 383f7d9bfd
fix: Registration of keymaps and associated errors. 2024-10-02 12:55:38 +02:00
Sofus Albert Høgsbro Rose 55235c7032
feat: Added handler bubble-up logic. 2024-10-02 11:24:48 +02:00
Sofus Albert Høgsbro Rose 12362b3cbd
deps: Added pytest-cov. 2024-10-02 10:25:56 +02:00
Sofus Albert Høgsbro Rose 606d336683
tests: Bootstrap and tested staticproperty.py 2024-10-02 10:21:11 +02:00
Sofus Albert Høgsbro Rose 514b49160e
refactor: Prepare for deeper refactor 2024-10-02 09:02:27 +02:00
Sofus Albert Høgsbro Rose ad548b8f03
refactor: Combing through spaghetti 2024-09-27 14:20:26 +02:00
Sofus Albert Høgsbro Rose e6ada5028d
refactor: Generate extension semantically 2024-09-26 13:33:25 +02:00
Sofus Albert Høgsbro Rose bac330ab2f
refactor: UV Transition and Rename #1 2024-09-26 10:23:17 +02:00
Sofus Albert Høgsbro Rose 4fc0528f6e
feat: mode solving / thesis demo ready 2024-09-05 17:20:46 +02:00
Sofus Albert Høgsbro Rose 81a71b2c47
feat: extensive improvements 2024-06-13 16:10:00 +02:00
Sofus Albert Høgsbro Rose b51c4f1889
feat: deep refactors / fixes
We refactored the entire `extra_sympy_units` into a (rather decently sized)
proper package, and changed its name.
Additionally, we've seperated the operation enums from the math nodes
themselves and moved them to dedicated `math_system` package, which is a
very big breath of fresh air.
I'll need a moment to fix a few typos, but incredibly, this architecture
is kind of "just works" (TM) at this point - not like the `FlowKind`
refactoring debacle...

To go with that, we've greatly streamlined domain handling in
`SimSymbol`.
We now track a symbolic set as the domain, which is very simple and
effective, but comes with some headaches too (`sympy` is fighting me,
grumble grumble...)
We've also managed to enforce a few unit-conversions in the operate
node, and revamp the entire operation validity detection (weirdly hard).

The big pain point at the moment is determining the image of functions
we apply, on the `sp.Set` domain of `SimSymbol`s. We can express this
trivially, but `sympy` simply doesn't care to evaluate it.

One can use `SetExpr` and `AccumBounds` to "sometimes work" for some set
kinds, but nothing nearly good enough for even our relatively humble
needs.
Let alone stuff like fourier.

We've ended up deciding to hard-code this part of the process by-operation.
With domain-specific knowledge and little bit of suffering, we can
manually ensure the output domain of every operation makes it to the
output symbol.

As for "why bother", well, the entire premise of a symbolic nodal math system
that is tolerable to use, requires checking the valid domain of the input.
We do wish it were optional, but eh.
2024-06-01 19:08:43 +02:00
Sofus Albert Høgsbro Rose 572d53f41e
fix: sim Value single realizations 2024-05-31 15:57:25 +02:00
Sofus Albert Høgsbro Rose 02e309db4d
feat: first fully parameterized simulation
We also believe we've fixed the crash related to read-during-write of
certain properties, by altering `BLField`'s invalidation to only
invalidate the non-persistent cache when it isn't suppressed.
This has no effect on the non-persistent cache, which is invalidated
correctly anyway during the write() that suppression is defined for;
however, by not stripping it away while that write() is being
implemented by Blender, CPython's builtin atomic `dict` operations
(especially `.get()`) manage to provide the thread-safety that we seem
to have been missing.

We tested the race-condition by the usual "drive everything from one
parameter", and weren't able to trigger one. Here's to hoping.

We've also started to become large fans of the "`Value` from `Func`"
principle, which both allows `Value` to pick up on fully realized
symbols in preceding flows, while also guaranteeing that we're not
maintaining duplicate code-paths.
However, the cost of the organizational reduction is that it is admittedly
slower (in the interactive hot-loop) than we'd like - in large part due
to having to realize a `FuncFlow` every time. However, certain
optimizations have a lot of potential:
- Reducing extraneous invalidations of `.lazy_func` on the `ExprSocket`,
  though some cleverness with `.output_sym` is needed.
- Pure performance work on all `Flow` objects. Surprisingly, they seem
  to mean a lot.
2024-05-31 14:20:50 +02:00
Sofus Albert Høgsbro Rose 3624d2ff45
feat: more lazy sim-design support
Making real headway.
2024-05-30 22:31:53 +02:00
Sofus Albert Høgsbro Rose c286d65193
fix: use-after-free in socket pruner
Also the usual batch of improvements.
Differentiability is misbehaving, intriguingly.
2024-05-30 21:39:48 +02:00
Sofus Albert Høgsbro Rose 38e70a60d3
feat: various sym-flow modifications 2024-05-30 18:41:06 +02:00
Sofus Albert Høgsbro Rose 830b316e01
feat: lazy boundconds aggregator
We also fixed the loose socket sorting bug w/some choice choices of
`inputs|outputs.move()`.
2024-05-27 21:06:38 +02:00
Sofus Albert Høgsbro Rose 624914f8cb
feat: lazy pml BC and better FuncFlow __or__ 2024-05-27 19:46:11 +02:00
Sofus Albert Høgsbro Rose d0615d0372
feat: bloch BC lazy support 2024-05-27 19:03:17 +02:00
Sofus Albert Høgsbro Rose 44a0ea95d3
feat: lazy absorber BC w/socket set fix 2024-05-27 18:19:57 +02:00
Sofus Albert Høgsbro Rose a3551c68b7
refactor: symbolic flow support
Massive boost.
Hard to quantify everything.

We're almost batch/inverse design!
- All units are now correct at realization (they weren't before), and
  several operations were simply wrong.
- Near-optimal "flow narrowing", which globally minimizes `DataChanged`
  even in the face of complex internal property dependencies.
- Sockets now cache `FlowKind` output themselves, with invalidation
  provided for and flow narrowing happening before it even enters the
  node flow.
- New FlowKind (`PreviewsFlow`) w/plotting, 3D, efficient caching, with
  all previewed nodes already adapted.
- Drastically prettified plot output, due to seaborn theme integration,
  dict-like data transport w/better labelling, etc. .
- Deep, reliable unit/unit dimension deduction and arithmetic for
  `PhysicalType` deduction using dimensional analysis on arbitrary
  expression operations.
- Fourier transform w/Nyquist limits as boundary conditions (preserving original shift).
- Vastly improved math node operation checks.
- Symbol constant node, integrating domain presumptions.
- Flow integration for ExprSocket, including for flow-dynamic capabilities and color.
- Viewer node got a big update, incl. live-display of sympy types (even
  matrix) and a few Tidy3D types, as well as seperating deeper options
  into a "Debug" mode
- Deeply streamlined symbolic flow, including
- **Differentiable Box structure**, w/generic lazy parameter support - only the start!
2024-05-27 16:48:27 +02:00
Sofus Albert Høgsbro Rose bcba444a8b
refactor: revamped symbolic flow (inaccurate unit conversions) 2024-05-24 16:01:23 +02:00
Sofus Albert Høgsbro Rose 353a2c997e
refactor: end-of-day commit (sim symbol flow for data import/export & inverse design) 2024-05-21 22:57:56 +02:00
Sofus Albert Høgsbro Rose dccf952ad3
refactor: renamed LazyValueFunc to Func 2024-05-21 09:04:05 +02:00
Sofus Albert Høgsbro Rose 84825c2642
refactor: Renamed LazyArrayRange to Range 2024-05-21 09:00:23 +02:00
Sofus Albert Høgsbro Rose f5d19abecd
feat: data file exporter node
As with many things, there seems to be an obvious convergent design
philosophy here wrt. data flow.
We should definitely be using `SimSymbol` for a lot more things; it
solves a lot of the pain points related to figuring out what on Earth
should go into the `InfoFlow` in which situations.

We desperately need to iron out the `*Flow` object semantics

The surprise MVP of the day is `Polars`.
What a gorgeous and fast dataframe library.
We initially wrote it off as being unsuited to multidimensional data,
but case in point, a whole lot of useful data can indeed be expressed as 2D.
For all of these cases, be it loading/saving or processing, `Polars`
is truly an ideal choice.

Work continues.
2024-05-21 08:51:26 +02:00
Sofus Albert Høgsbro Rose 0f2f494868
feat: implemented data loading w/new math ops
We implemented a node to load various kinds of data, notably `.npy`,
`.txt`, `.txt.gz`, and `.csv`. The `DataFileImporterNode` really should
expose some settings for setting name/mathtype/physical type/unit of
each unit, and/or treating a column from 2D data as index coordinates.
But the nuances of doing this in a manner general enough to deal with
!=2D data was a lot, and we needed similar abilities in the general math
system anyway.

So, we delved back into the `FilterMathNode` and a little into the
`TransformMathNode`. Fundamentally, a few difficult operations came out
of this:

- Filter:SliceIdx: Slice an array using the usual syntax, as baked into the
  function.
- Filter:PinIdx: Pin an axis by its actual index.
- Filter:SetDim: Set the `InfoFlow` index coordinates of an axis to a specific,
  loose-socket provided 1D array, and use a common symbol to set the
  name+physical type (and allow specifying an appropriate unit).
- Transform:IntDimToComplex: Fold a last length-2 integer-indexed axis
  into a real output type, which removes the dimension and produces a
  complex output type. Essentially, this is equivalent to folding it as
  a vector and treating the `R^2` numbers as real/imaginary, except this
  is more explicit.

By combining all of these, we managed to process and constrain the medium data to
be a well-suited, unit-aware (**though not on the output (yet)**) `wl->C` tensor.
In particular, the slicing is nice for avoiding discontinuities.

Workflow-wise, we'll see how important these are / what else we might
want. Also, it turns out Blender's text editor is really quite nice for
light data-text viewing.
2024-05-19 18:04:58 +02:00
Sofus Albert Høgsbro Rose a66a28da27
feat: transform node w/sane `DataChanged`-chaining
We also implemented a JIT-based `rescale` for `ArrayFlow`, where the
same function as for `LazyArrayRangeFlow` passes through and can do an
arbitrary symbolic shift/rescale/order-preserving whatever.
To make this fast for `ArrayFlow`, we utilize a "test" variable `a`,
put it through the function and rescale/strip its units, then `lambdify`
it and broadcast the function onto `ArrayFlow.values`.

It's an immensely clean way of working.
The `lambdify` seems fast, interestingly, but ideally we would of course
also cache that somehow.

Some details remain:

- Fourier Transform index bounds / length are currently not presumed
  known before actually computing them; it's unclear what's best here,
  as the design cross-section of physical expectation, mathematical
  correctness, and ease of implementation (especially trying to keep
  actual data out of the `InfoFlow` hot-path). For now, the `0..\infty`
  bounds **will probably break** the `Viz` node.
- We've reverted the notion that particular `sim_symbol`s must have a
  unit pre-defined, which causes a little more complexity for nodes like
  `TemporalShape`. This question is going to need resolving
- The whole `InfoFlow` object really seems to be at the heart of certain
  lagginess when it comes to the math system. It, together with the
  index representations, would benefit greatly from a principled
  refactor.
- The `Viewer` node is broken for 3D preview; see #70.

Closes #59. Progress on #54.
2024-05-19 09:06:33 +02:00
Sofus Albert Høgsbro Rose 39747e2d68
fix: map node up to current standards 2024-05-18 18:42:54 +02:00
Sofus Albert Høgsbro Rose 532f0246b5
fix: operate node and "remembered" math ops
We reimplemented the OperateMathNode entirely, and it should now be
relatively to entirely robust. It works based on an enum, just like
`MapMathNode`, which enormously simplifies the node itself. Note that
we're not quite done with sympy+jax implementations of everything, nor
validating that all operations have an implementation valid for all the
`InfoFlow`s that prompted it, but, nonetheless, such fixes should be
easy to work out as we go.

We also finally managed to implement "remembering". In essence,
"remembering" causes `FlowPending` to not reset the dynamic enums in
math / extract nodes. The implementation is generally per-node, and a
bit of boilerplate, but it seems quite robust on the whole - allowing
one to "keep a node tree around" even after removing simulation data.

Extract of sim data is of course more leniant, as it also "remembers"
when there is `NoFlow`. Only new `Sim Data` will reset the extraction
filter enum.

Caches are kept (not memory efficient, but convenient for me), but
only `FlowPending` is actually passed along the `InfoFlow`. This is the
semantics we want - having to deal with this all is a tradeoff of
having data-driven enums, but all in all, it still feels like the right
approach.
2024-05-18 18:02:15 +02:00
Sofus Albert Høgsbro Rose 035d8971f3
fix: local invalidation chains w/extract fixes 2024-05-18 12:09:37 +02:00
Sofus Albert Høgsbro Rose e889d20284
fix: make extract node use `LazyValueFunc` 2024-05-18 09:56:34 +02:00
Sofus Albert Høgsbro Rose 785c6f764c
fix: GN update for custom geonodes
We also implement `BLField` support for Blender `IDStruct` types.

There's a crash haunting us specifically with the cylinder array. Other
primitives (esp. ring) work just fine, as does previews of nested-linked
node groups. The crash triggers specifically when the file is saved and
reloaded, whereafter all the `load_post` handlers run fine (like -
extremely fine, we can perfectly access and dereference all of the node
groups, seemingly all the objects, etc.). Then, crash.

We're also discovering that `id_properties_ui` is completely useless for
after-the-fact updating of custom properties within sockets. Which is a
great surprise, and I have trouble thinking it's on purpose - the data
is stored somewhere, after all. All the forced updates/redraws/etc. in
the world don't seem to change this.

We may have to go back to the drawing board with dynamically-updated
min/max. The entire infrastructure with `SocketDef` altering sockets
after creation is entirely, _violently_ unsuited to do a static
modification. But the bare fact is, the dynamic modification methods are
falling short. It's kind of important stuff, this stuff.
2024-05-17 12:39:11 +02:00
Sofus Albert Høgsbro Rose 7ac6b615de
feat: cylinder primitive & fixed GN previews
Re-added the cylinder primitive (it's quite important), including an
appropriate asset `.blend`.

Also finished (I think?) a quick cleanup round from the enhanced
`ManagedBLModifier`, which now owns the mesh and doesn't need a seperate
`ManagedBLMesh`.
This seems faster for some reason, but more importantly, it simplifies a
bunch of the previewing code.

Finally, fixed a small issue with the extract data node where its
`ExprSocket` still had the `FlowKind.Array`, causing it to not be able
to be connected to the whole math system. So, yeah.
2024-05-16 19:15:11 +02:00
Sofus Albert Høgsbro Rose 2d26ea6ce8
feat: merged temporal shapes w/symbolic envelope
We now have a single node for all temporal shapes, which is extremely
usable. Note that we found a bug where input socket caching seems to
survive changes to loose inputs / socket set-driven alterations, which
prevents output from being able to switch with the socket set. We'll
make an issue for it whenever convenient.

Work also continued very briskly with the `SimSymbol` abstraction, which
is really, really working out.

Closes #67.
2024-05-16 18:02:01 +02:00
Sofus Albert Høgsbro Rose e51ec8f43f
fix: renamed Tidy3D Web Runner to Importer
This provides consistency in the interface.
2024-05-16 13:15:39 +02:00
Sofus Albert Høgsbro Rose 060f54bd94
feat: expr constant w/viz fixes
Driven solely by the Expr socket, we've completely replaced the
dedicated NumberConstant and PhysicalConstant nodes. We also
demonstrated symbolic variable operations w/visualization and
end-node realization, validating that this approach to Expr sockets is
key.

We prepared for a new `BLPropType`, namely dynamic lists of dataclasses,
which will represent dynamic user-adjustable lists. This is the only
proper UI design for declaring symbols directly in an Expr node.
For now, we'll do without symbols, but will be a core feature for design
space exploration (aka. batch-run a bunch of sims), and for inverse
design.

Else, a few fixes: Naming of `ManagedBLImage` was updated, the expr
socket info display was fixed to display even when only output
information is available,

A few TODOs remain in the Expr Constant, but they are less important
before dynamic symbol declarations are in place.
2024-05-16 13:01:37 +02:00
Sofus Albert Høgsbro Rose 92be84ec8a
feat: perm monitor, enh. monitors, sim node naming
We greatly enhanced the field, flux monitors, and added the permittivity
monitor.

mobj naming is still a bit borked; we need a node-tree bound namespace
to go further with it.
For now, don't duplicate nodes, and don't use multiple node trees.
2024-05-16 11:06:18 +02:00
Sofus Albert Høgsbro Rose af358a4d32
fix: reordered init to suppress warnings and unit-conversion fix 2024-05-15 14:08:00 +02:00
Sofus Albert Høgsbro Rose c9936b8942
feat: big refactor to fight fundamental crashes
Some rather foundational things were fundamentally broken, especially related to the initialization procedures of fields / cached properties.

- We completely revamped `bl_cache`, fixing many to-be-discovered bugs.
- We completely streamlined `BLField` property logic into reusable
  `bl_cache.BLProp` and `bl_cache.BLPropType`.
- We implemented `BLInstance` superclass to handle ex. deterministic
  persistance of dynamic enum items, and other nuanced common
  functionality that was being duplicated raw.
- We implemented inter `cached_bl_property` / `BLField` dependency
  logic, including the ability to invalidate dynamic enums without
  @on_value_changed logic. This **greatly** simplifies a vast quantity
  of nodes that were traditionally very difficult to get working due to
  the sharp edges imposed by needing manual invalidation logic.
- We gave `ExprSocket` a significant usability upgrade, including
  thorough parsing logic in the `SocketDef`.

It's not that existing nodes are as such broken, but their existing bugs
are now going to cause problems a lot faster.
Which is a good thing.

BREAKING CHANGE: Closes #13. Closes #16. Big work on #64. Work on #37.
2024-05-15 12:37:38 +02:00
Sofus Albert Høgsbro Rose 929fb2dae9
feat: implemented `GaussianBeam` source
This node is good for approximating a simple laser.
2024-05-07 12:39:17 +02:00
Sofus Albert Høgsbro Rose 9f8ff33e4f
feat: implement plane wave node w/viz
the visualization of Tidy3D's spherical coordinates was exceptionally harsh. See #63 for more information.

Closes #63.
2024-05-06 22:11:05 +02:00
Sofus Albert Høgsbro Rose 0fbd3752b3
feat: real-time videos of simulated EM fields
We did it! We have a `Scene` node which outputs the current frame/time
mapped to a friendlier unit (like `ps`) and takes an integer FPS.

A `frame_post` handler in Blender makes sure to run `DataChanged` on any
`Scene` nodes in any active node trees (a notion we added with this
commit), which causes a `@property` to effectively masquerade as a
"real" property. Thus, all the caching/other stuff like that has its
semantics preserved, so long as the invalidated `@property` stays.

We can probably optimize the handler - it linearly scans all nodes in
all trees. A bit suboptimal. We can also think more about more outputs
and such. Still, a fine start, and very cool with videos of EM fields
doing their thing! (Of course, one can plug this into anything, not just
the time-fixing part of an Expr chain).

Closes #17.
2024-05-05 17:57:07 +02:00
Sofus Albert Høgsbro Rose 084ae632f1
fix: fixed bugs preventing end-to-end sim
Several bugs/gotchas/papercuts have been fixed, which together made it really hard

Closes #47 #48 #49 #50 #51 #52.
2024-05-05 14:27:02 +02:00
Sofus Albert Høgsbro Rose 999d601c49
fix: fixed web runner import 2024-05-05 10:00:45 +02:00
Sofus Albert Høgsbro Rose b0db0ca072
fix: fix import and remove some TODOs
We're slowly migrating all the random sprinkled TODOs into actual, trackable issues, so we actually have a change of keeping an overview.
2024-05-05 09:51:17 +02:00