so-rose
/
oscillode
1
0
Fork 0
Code Issues 50 Pull Requests Packages Projects Releases Wiki Activity

plan: Revamped TODO w/final planning

main
Sofus Albert Høgsbro Rose 2024-04-27 19:49:46 +02:00
parent 0599eed2b2
commit 8012415278
Signed by: so-rose
GPG Key ID: AD901CB0F3701434
5 changed files with 208 additions and 293 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

402
TODO.md
View File

@ -1,93 +1,105 @@
# Acute Tasks
- [x] Implement Material Import for Maxim Data
- [x] Implement Robust DataFlowKind for list-like / spectral-like composite types
- [x] Unify random node/socket caches.
- [x] Revalidate cache logic
- [x] Finish math system
- [ ] Finish the "Low-Hanging Fruit" Nodes
- [ ] Move preview GN trees to the asset library.
# Working TODO
- [ ] Wave Constant
- [ ] Fix many problems by persisting `_enum_cb_cache` and `_str_cb_cache`.
# Nodes
## Analysis
- [x] Extract
- [ ] Implement "saved" state for info, and provide the user an indicator that state has been saved w/a button to reset (the state should also be reset when plugging a new data thing in)
- [x] Viz
- [ ] Implement Info-driven planar projection of pixels onto managed image empty.
- [ ] Live-slice 2D field values onto user-controlled image empty from 2D field.
- [ ] SocketType-based visualization support.
- [ ] Pol SocketType: 2D elliptical visualization of Jones vectors.
- [ ] Pol SocketType: 3D Poincare sphere visualization of Stokes vectors.
- [x] Math / Map Math
- [ ] Remove "By x" socket set let socket sets only be "Function"/"Expr"; then add a dynamic enum underneath to select "By x" based on data support.
- [ ] Filter the operations based on data support, ex. use positive-definiteness to guide cholesky.
- [ ] Implement support for additional symbols via `Expr`.
- [x] Math / Filter Math
- [ ] Math / Reduce Math
- [ ] Math / Operate Math
- [x] Math / Operate Math
- [ ] Remove two-layered dropdown; directly filter operations and use categories to seperate them.
- [ ] Implement Expr socket advancements to make a better experience operating between random expression-like sockets.
## Inputs
- [x] Wave Constant
- [x] Implement export of frequency / wavelength array/range.
- [x] Unit System
- [ ] Implement presets, including "Tidy3D" and "Blender", shown in the label row.
- [ ] Fix the LazyValueRange (again!)
- [ ] Document
- [x] Scene
- [ ] Implement export of scene time via. Blender unit system.
- [ ] Implement optional scene-synced time exporting, so that the simulation definition and scene definition match for analysis needs.
- [x] Constants / Scientific Constant
- [x] Create `utils.sci_constants` to map `scipy` constants to `sympy` units.
- [x] Utilize `utils.sci_constants` to make it easy for the user to select appropriate constants with two-layered dropdowns.
- [x] Constants / Expr Constant
- See IDEAS.
- [x] Constants / Number Constant
- [ ] Fix non-integer sockets
- [ ] Constants / Vector Constant
- [ ] Constants / Physical Constant
- [ ] Pol: Elliptical viz as 2D plot.
- [ ] Pol: Poincare sphere viz as 3D GN.
- [x] Constants / Scientific Constant
- [ ] Nicer (boxed?) node information, maybe centered headers, in a box, etc. .
- [x] Constants / Unit System Constant
- [ ] Implement presets, including "Tidy3D" and "Blender", shown in the label row.
- [x] Constants / Blender Constant
- [ ] Fix it!
- [ ] Web / Tidy3D Web Importer
- [ ] Have a visual indicator for the current download status, with a manual re-download button.
- [ ] Fix the check of folders, actually, just fix `tdcloud` in general!
- [ ] Have a visual indicator for the download status of the currently selected task, as well as its data size.
- [ ] If a task is "selected", lock the cloud task socket, so other tasks can't be selected. While that lock is active, expose a real "download" button. Also make the loose output socket and put out a `FlowPending` until the download is available.
- [ ] A manual download button and seperate re-download button (maybe on the side, round reload boi).
- [ ] An option to pack the data into the blend, with overview of how much data it will take (Base85/base64 has overhead).
- [ ] Default limits for caching/packing.
- [ ] Support importing batched simulations and outputting an `Array` of SimData.
- [x] File Import / Material Import
- [x] Dropdown to choose import format
- MERGED w/TIDY3D FILE IMPORT
- [x] File Import / Tidy3D File Import
- [x] HDF and JSON file support, with appropriate choice of loose output socket.
- [ ] File Import / Array File Import
- [ ] File Import / Data File Import
- [ ] Implement `FlowKind.LazyValueFunc` that plays the loading game.
- [ ] Implement `FlowKind.Info` which lets the user describe the data being loaded, for proper further processing.
- [ ] Implement unit system input to guide conversion from numpy data type.
- [ ] Implement datatype dropdown to guide format from disk, prefilled to detected.
- [ ] Implement `FlowKind.Array` that just runs the `LazyValueFunc` as usual.
- [ ] Standardize 1D and 2D array loading/saving on numpy's savetxt with gzip enabled.
- [ ] Implement unit system input to guide conversion from numpy data type.
- [ ] Implement datatype dropdown to guide format from disk, prefilled to detected.
- [ ] Implement a LazyValue to provide a data path that avoids having to load massive arrays every time always.
- [x] File Import / Tidy3D File Import
## Outputs
- [x] Viewer
- [ ] Remove image preview when disabling plots.
- [x] Auto-enable 3D preview when creating.
- [ ] Consider a "debug" mode
- [ ] Auto-enable plot when creating.
- [ ] Test/support multiple viewers at the same time.
- [ ] Pop-up w/multiline string as alternative to console print.
- [ ] Handle per-tree viewers, so that switching trees doesn't "bleed" state from the old tree.
- [ ] BUG: CTRL+SHIFT+CLICK not on a node shows an error; should just do nothing.
- [x] Web Export / Tidy3D Web Exporter
- This is an extraordinarily nuanced node, and will need constant adjusting if it is to be robust.
- [ ] Run checks on-demand, and require they be run before the sim can be uploaded. If the simulation changes, don't
- [ ] Support doing checks in a seperate process.
- [ ] We need better ways of doing checks before uploading, like for monitor data size. Maybe a SimInfo node?
- [ ] Implement "new folder" feature w/popup operator.
- [ ] Implement "delete task" feature w/popup confirmation.
- [ ] We need to be able to "delete and re-upload" (or maybe just delete from the interface).
- [ ] Accept `Array` of simulations, and upload them as `Batch`.
- [x] File Export / JSON File Export
- [ ] File Import / Tidy3D File Export
- [ ] Reevaluate its purpose.
- [ ] File Export / Tidy3D File Export
- [ ] Implement HDF-based export of Tidy3D-exported object (which includes ex. mesh data and such)
- [ ] Also JSON (but indicate somehow that ex. mesh data doesn't come along for the ride).
- [ ] File Export / Array File Export
- [ ] File Export / Data File Export
- [ ] Implement datatype dropdown to guide format on disk.
- [ ] Implement unit system input to guide conversion to numpy data type.
- [ ] Standardize 1D and 2D array loading/saving on numpy's savetxt with gzip enabled.
## Viz
- [x] Monitor Data Viz
- [x] Implement dropdown to choose which monitor in the SimulationData should be visualized (based on which are available in the SimulationData), and implement visualization based on every kind of monitor-adjascent output data type (<https://docs.flexcompute.com/projects/tidy3d/en/latest/api/output_data.html>)
- [ ] Project field values onto a plane object (managed)
## Sources
- [x] Temporal Shapes / Gaussian Pulse Temporal Shape
- [x] Temporal Shapes / Continuous Wave Temporal Shape
- [ ] Temporal Shapes / Symbolic Temporal Shape
- [ ] Specify a Sympy function to generate appropriate array based on
- [ ] Temporal Shapes / Array Temporal Shape
- [ ] Temporal Shapes / Data Temporal Shape
- [x] Point Dipole Source
- [ ] Use a viz mesh, not empty (empty doesn't play well with alpha hashing).
- [ ] Plane Wave Source
- [x] Implement an oriented vector input with 3D preview.
- [ ] **IMPORTANT**: Fix the math so that an actually valid construction emerges!!
- [ ] Uniform Current Source
- [ ] TFSF Source
@ -95,14 +107,17 @@
- [ ] Gaussian Beam Source
- [ ] Astigmatic Gaussian Beam Source
- [ ] Mode Source
- [ ] Array Source / EH Array Source
- [ ] Array Source / EH Equiv Array Source
- [ ] EH Array Source
- [ ] EH Equiv Array Source
## Mediums
- [x] Library Medium
- [ ] Implement frequency range output (listy)
- [ ] Implement frequency range output (listy), perhaps in the `InfoFlow` lane?
- [ ] Implement dynamic label.
- [ ] DataFit Medium
- [ ] Implement by migrating the material data fitting logic from the `Tidy3D File Importer`, except now only accept a `Data` input socket, and rely on the `Data File Importer` to do the parsing into an acceptable `Data` socket format.
- [ ] Save the result in the node, specifically in a property (serialized!) and lock the input graph while saved.
- [ ] PEC Medium
- [ ] Isotropic Medium
- [ ] Anisotropic Medium
@ -117,55 +132,48 @@
- [ ] Non-Linearity / Two-Photon Absorption Non-Linearity
- [ ] Non-Linearity / Kerr Non-Linearity
- [ ] Space/Time epsilon/mu Modulation
## Structures
- [ ] BLObject Structure
- [x] GeoNodes Structure
- [x] Rewrite the `bl_socket_map.py`
- [x] Use the modifier itself as memory, via the ManagedObj
- [x] Rewrite to use unit systems properly.
- [ ] Implement a panel system, to make GN trees with a ton of inputs (most of which are not usually needed) actually useful.
- [ ] Propertly map / implement Enum input sockets to the GN group.
- [ ] Implement a panel system, either based on native GN panels, or description parsing, or something like that.
- [ ] Primitive Structures / Line Structure
- [ ] Primitive Structures / Plane Structure
- [x] Primitive Structures / Box Structure
- [x] Primitive Structures / Sphere Structure
- [ ] Primitive Structures / Cylinder Structure
- [ ] Primitive Structures / Ring Structure
- [ ] Primitive Structures / Capsule Structure
- [ ] Primitive Structures / Cone Structure
## Monitors
- [x] E/H Field Monitor
- [x] Field Power Flux Monitor
- [ ] \epsilon Tensor Monitor
- [ ] Diffraction Monitor
- [ ] Axis-aligned planar 2D (pixel)
- [ ] Projected E/H Field Monitor / Cartesian Projected E/H Field Monitor
- [ ] Use to implement the metalens: <https://docs.flexcompute.com/projects/tidy3d/en/latest/notebooks/Metalens.html>
- [ ] Projected E/H Field Monitor / Angle Projected E/H Field Monitor
- [ ] Projected E/H Field Monitor / K-Space Projected E/H Field Monitor
- [ ] Modal Nodes
- Spatial+frequency feature monitoring. An EM field can be decomposed into using a specially configured solver, which can be used to look for very particular kinds of effects by constraining investigations of a solver result to filter out everything that isn't these particular modes aka. features. Kind of a fourier-based redimensionalization, almost).
## Simulations
- [x] FDTDSim
- [x] Sim Domain
- [ ] By-Medium batching of Structures when building the td.Simulation object, which can have significant performance implications.
- [ ] Primitive Structures / PolySlab Structure
## Bounds
- [x] Boundary Conds
- [ ] Boundary Cond / PML Bound Face
- [ ] Dropdown for "Normal" and "Stable"
- [ ] Boundary Cond / PEC Bound Face
- [ ] Boundary Cond / PMC Bound Face
- [ ] Boundary Cond / Bloch Bound Face
- [ ] Boundary Cond / Periodic Bound Face
- [ ] Implement "simple" mode aka "periodic" mode in Tidy3D
- [ ] Boundary Cond / Absorbing Bound Face
## Monitors
- [x] EH Field Monitor
- [ ] Method of setting `inf` on dimensions - use a `ManyEnum` maybe to select the injection axis, and let that set the $0$.
- [ ] Revamp the input parameters.
- [x] Power Flux Monitor
- [ ] Permittivity Monitor
- [ ] Diffraction Monitor
- [ ] Projected E/H Field Monitor / Cartesian Projected E/H Field Monitor
- [ ] Use to implement the metalens: <https://docs.flexcompute.com/projects/tidy3d/en/latest/notebooks/Metalens.html>
- [ ] Projected E/H Field Monitor / Angle Projected E/H Field Monitor
- [ ] Projected E/H Field Monitor / K-Space Projected E/H Field Monitor
## Simulations
- [x] FDTDSim
- [ ] By-Medium batching of Structures when building the td.Simulation object, which can have significant performance implications.
- [x] Sim Domain
- [ ] Sim Grid
- [ ] Sim Grid Axes / Auto Sim Grid Axis
- [ ] Sim Grid Axes / Manual Sim Grid Axis
@ -174,8 +182,10 @@
## Utilities
- [ ] Separate
- [ ] Use generic Expr socket mode to combine numerical types into either Expr or Data socket.
- [x] Combine
- [x] Implement concatenation of sim-critical socket types into their multi-type
- [ ] Use generic Expr socket mode to combine numerical types into either Expr or Data socket.
- [ ] Explicit about lower structures taking precedence.
@ -183,34 +193,30 @@
- [ ] Tests / Monkey (suzanne deserves to be simulated, she may need manifolding up though :))
- [ ] Tests / Wood Pile
- [ ] Structures / Primitives / Line
- [ ] Structures / Primitives / Plane
- [x] Structures / Primitives / Box
- [x] Structures / Primitives / Sphere
- [ ] Structures / Primitives / Cylinder
- [x] Structures / Primitives / Ring
- [ ] Structures / Primitives / Capsule
- [ ] Structures / Primitives / Cone
- [ ] Structures / Arrays / Square
- [ ] Structures / Arrays / Square-Hole
- [ ] Structures / Arrays / Cyl
- [ ] Structures / Arrays / Cyl-Hole
- [x] Structures / Arrays / Box
- [x] Structures / Arrays / Sphere
- [ ] Structures / Arrays / Box
- [ ] Structures / Arrays / Sphere
- [ ] Structures / Arrays / Cylinder
- [x] Structures / Arrays / Ring
- [ ] Structures / Arrays / Capsule
- [ ] Structures / Arrays / Cone
- [ ] Array / Square Array **NOTE: Ring and cylinder**
- [ ] Array / Hex Array **NOTE: Ring and cylinder**
- [ ] Hole Array / Square Hole Array: Takes a primitive hole shape.
- [ ] Hole Array / Hex Hole Array: Takes a primitive hole shape.
- [ ] Cavity Array / Hex Array w/ L-Cavity
- [ ] Cavity Array / Hex Array w/ H-Cavity
- [ ] Structures / Hex Arrays / Cyl
- [ ] Structures / Hex Arrays / Box
- [ ] Structures / Hex Arrays / Sphere
- [ ] Structures / Hex Arrays / Cylinder
- [x] Structures / Hex Arrays / Ring
- [ ] Crystal Sphere Lattice / Sphere FCC Array
- [ ] Crystal Sphere Lattice / Sphere BCC Array
- [ ] Structures / Cavity Arrays / L-Cavity Cylinder
- [ ] Structures / Cavity Arrays / H-Cavity Cylinder
- [ ] Structures / Lattice Arrays / FCC Sphere
- [ ] Structures / Lattice Arrays / BCC Sphere
@ -233,6 +239,9 @@
- [x] String
- [x] File Path
- [x] Color
- [x] Expr
- [ ] Implement node-driven support for dynamic symbols.
- [ ] Implement compatibility with sockets that fundamentally do produce expressions, especially Physical sockets.
## Number
- [x] Integer
@ -279,7 +288,10 @@
## Tidy3D
- [x] Cloud Task
- [ ] Implement switcher for API-key-having config filconfig file vs. direct entry of API key. It should be auto-filled with the config file when such a thing exists.
- [ ] Move API checking out of the socket, and don't re-prompt for a key if the config file exists.
- [ ] Remove the existing task selector when making a new task.
- [ ] Implement "new folder" feature w/popup operator.
- [ ] Implement "delete task" feature w/popup confirmation.
## Physical
- [x] Unit System
@ -350,16 +362,18 @@
# Internal / Architecture
## IDEAS
- [ ] Socket "guarding" - let nodes influence the dynamic capabilities of sockets to prevent links (with a `self.report` explanation) to an output socket that won't yet produce a value.
- [ ] Prevents some uses of loose sockets (we want less loose sockets!)
## CRITICAL
- [ ] Rethink the way that loose sockets are replaced, specifically with respect to deterministic ordering.
- Currently order is not guaranteed. This is causing problems.
## User-Facing Errors and Legal Considerations
- [ ] `log.error` should invoke `self.report` in some Blender operator - used for errors that are due to usage error (which can't simply be prevented with UX design, like text file formatting of import), not due to error in the program.
- [ ] License header UI for MaxwellSimTrees, to clarify the AGPL-compatible potentially user-selected license that trees must be distributed under.
- [x] Document the node tree cache semantics thoroughly; it's a VERY nuanced piece of logic, and its invariants may not survive Blender versions / the author's working memory
- [ ] Start standardizing nodes/sockets w/individualized SemVer
- [ ] A "CitationsFlow" FlowKind which simply propagates citations.
- [ ] Implement standardization of nodes/sockets w/individualized SemVer
- Perhaps keep node / socket versions in a property, so that trying to load an incompatible major version hop can error w/indicator of where to find a compatible `blender_maxwell` version.
- Integrate w/BLField, to help the user manage addon updates that would break their tree.
## Documentation
- [ ] Make all modules available
@ -368,16 +382,12 @@
- [ ] Comb through and finish `__doc__`s.
## Performance
- [ ] The GN value pushing currently does an expensive from-sympy conversion for all GN attributes, making it very slow.
- Generally, the issue is that we can't compare the pushed value to the existing value without a conversion.
- Also, sympy expressions can't be hashed (by default), and `str()` may be just as slow, so a simple `@lru_cache` is no good.
- **One approach** is a `HashableSympyExpr` class, which would use ex. atoms and values in `SympyExpr` to let us `@lru_cache` the sympy conversion.
- **Another approach** is to "just" make the `scale_to_unit` function faster (we should profile).
- Presumably, the scaling factors between units could be eagerly cached, then the numerical part of the expression could be used to avoid `convert_to` calls.
- Without the list format of `(np.array, spu.Quantity)`, `sp.Matrix` can't be performantly handled like this.
- [ ] Optimize GN value pushing w/sympy expression hashing.
## Style
- [ ] Color of nodes should be a nice blue, eh?
Header color style can't be done, unfortunately. Body color feels unclean, so nothing there for now.
- [ ] Node icons to denote preview/plot state.
## Registration and Contracts
- [ ] Refactor the node category code; it's ugly.
@ -396,47 +406,30 @@
- [ ] Document the `tdcloud` service thoroughly and open a GitHub discussion about `td.web` shortcomings.
## Node Base Class
- [ ] Implement a new socket type for preview-only parameters
- [ ] When used, the node base class should expose a toggle
- [ ] Instead of mangling props, we can instead reuse all of the socket-based code, while also enabling composability of previews.
- [ ] Custom `@cache`/`@lru_cache`/`@cached_property` which caches by instance ID (possibly based on `beartype` or `pydantic`).
- The problem to solve is performance vs. persistence.
- [ ]
- [ ] Implement by-category sorting of loose sockets with 'move' method on `node.inputs`/`node.outputs`.
- Currently order is not guaranteed
- [ ] When presets are used, if a preset is selected and the user alters a preset setting, then dynamically switch the preset indicator back to "Custom" to indicate that there is no active preset
- [ ] Re-engineer "presets" to use an Enum of some kind.
## Events
- [x] Mechanism for selecting a blender object managed by a particular node.
- [ ] Standard way of triggering the selection
- [ ] Mechanism for ex. specially coloring a node that is currently participating in the preview.
- [ ] Custom callbacks when deleting a node (in `free()`), to ex. delete all previews with the viewer node.
- [ ] When a Blender object is selected, select the node that owns its ManagedObj.
- [ ] Node button / shortcut / something to select the ManagedObj owned by a node.
## Socket Base Class
- [ ] Second-generation listy, based on a `DataFlowKind.ValueListy`, `DataFlowKind.ValueRange`, etc. to encode the presence of special logic.
- This is key to allow special handling, as "just give me a `list[]` of `sympy` objects" is an exceptionally non-performant and brittle thing.
- [ ] Implement capability system, which defaults to exactly matching the type.
- [ ] Make `to_socket`s no-consent to new links from `from_socket`s of incompatible Capability.
- [ ] Add Capabilities needed mainly in cases where we need `Any` currently.
- [ ] Collect `SocketDef` objects like we do with `BL_REGISTER`, without any special mojo sauce.
## Many Nodes
- [ ] Implement "Steady-State" / "Time Domain" on all relevant Monitor nodes
- [ ] (?) Dynamic `bl_label` where appropriate (ex. "Library Medium" becoming "Au Medium")
- [ ] Implement LazyValue, including LazyParamValue on a new class of constant-like input nodes that really just emit ex. sympy variables.
- [ ] Medium Features
- [ ] Accept spatial field. Else, spatial uniformity.
- [ ] Accept non-linearity. Else, linear.
- [ ] Accept space-time modulation. Else, static.
- [ ] Modal Features
- [ ] ModeSpec, for use by ModeSource, ModeMonitor, ModeSolverMonitor. Data includes ModeSolverData, ModeData, ScalarModeFieldDataArray, ModeAmpsDataArray, ModeIndexDataArray, ModeSolver.
- ModeSpec, for use by ModeSource, ModeMonitor, ModeSolverMonitor. Data includes ModeSolverData, ModeData, ScalarModeFieldDataArray, ModeAmpsDataArray, ModeIndexDataArray, ModeSolver.
## Many Sockets
- [ ] Implement constrained SympyExpr checks all over the place.
## Development Tooling
- [ ] Implement `pre-commit.
- [ ] Pass a `mypy` check
- [ ] Pass all `ruff` checks, including `__doc__` availability.
- [ ] Implement `pre-commit.
- [ ] Add profiling support, so we can properly analyze performance characteristics.
- Without a test harness, or profile-while-logging, there may be undue noise in our analysis.
- [ ] Simple `pytest` harnesses for unit testing of nodes, sockets.
@ -454,41 +447,24 @@
- We might still want/need the jax based stuff after; volume geonodes aren't finalized.
## Packaging
- [ ] Popup to install dependencies after UI is available (possibly with the help of the `draw()` function of the `InstallPyDeps` operator)
- [ ] Use a Modal and multiline-text-like construction to print `pip install` as we install dependencies, so that the user has an idea that something is happening.
- [ ] Test lockfile platform-agnosticism on Windows
## BLCache
- [ ] Replace every raw property with `BLField`.
- [ ] Add matrix property support: https://developer.blender.org/docs/release_notes/3.0/python_api/#other-additions
- [ ] Fix many problems by persisting `_enum_cb_cache` and `_str_cb_cache`.
- [ ] Docstring parser for descriptions.
- [ ] Method of dynamically setting property options after creation, using `idproperty_ui_data`
# BUGS
We're trying to do our part by reporting bugs we find!
This is where we keep track of them for now.
This is where we keep track of them for now, if they're not covered by the above listings.
## Blender Maxwell Bugs
- [ ] Detaching data chained into Viz node makes for a very laggy error, as non-implemented LazyValueFunc suddenly can't propagate live into the Viz node.
- [ ] Need to clear invalid searched StrProperties on copy
- [ ] Enabled 3D preview is really slow for some reason when working with the math nodes.
- [ ] BUG: CTRL+SHIFT+CLICK not on a node shows an error; should just do nothing.
- [ ] Slow changing of socket sets / range on wave constant.
- [ ] API auth shouldn't show if everything is fine in Cloud Task socket
- [ ] Cloud task socket loads folders before its node shows, which can be slow (and error prone if offline)
- [ ] Dispersive fit is slow, which means lag on normal operations that rely on the fit result - fit computation should be integrated into the node, and the output socket should only appear when the fit is available.
- [ ] Numerical, Physical Constant is missing entries
- [ ] Numerical, Physical Constant is missing entries
BROKE NODES
- [ ] Numerical constant doesn't switch types
- [ ] Blender constant is inexplicably mega laggy
- [ ] Web importer is just wonky in general
- [ ] JSON File exporter is having trouble with generic types (is that bad?)
- [ ] Extact Data needs flux settings
- [ ] Point dipole still has no preview
- [ ] Plane wave math still doesn't work and it has no preview
- [ ] Monitors need a way of setting infinite dimensions
## Blender Bugs
Reported:
@ -511,87 +487,39 @@ Unreported:
- Let's have operator `poll_message_set`: https://projects.blender.org/blender/blender/commit/ebe04bd3cafaa1f88bd51eee5b3e7bef38ae69bc
- Careful, Python uses user site packages: <https://projects.blender.org/blender/blender/commit/72c012ab4a3d2a7f7f59334f4912402338c82e3c>
- Our modifier obj can see execution time: <https://projects.blender.org/blender/blender/commit/8adebaeb7c3c663ec775fda239fdfe5ddb654b06>
- We found the translation callback! https://projects.blender.org/blender/blender/commit/8564e03cdf59fb2a71d545e81871411b82f561d9
- This can update the node center!!
- [x] Optimize the `DataChanged` invalidator.
- [ ] Optimize unit stripping.
## IDEAS
- [ ] Depedencies-gated addon preferences.
- [ ] Preferences-based specification/alteration of default unit systems for Tidy3D and Blender.
- [ ] Preferences-based specification/alteration of Tidy3D API key, so we can factor away all the `prelock` bullshit.
- [ ] Subsockets
- We need Exprs to not be so picky.
- All the sympy-making nodes should be subsockets of Expr, so that you can plug any socket that should work with Expr into Expr.
- When it comes to Data, any Expr that produces an array-like output from its `LazyValueFunc` should be deemed compatible (as in, the Expr may plug into a Data socket).
- Specifically, that means the presence of a well-defined `Info`, as well as `jax` compatibility.
- [ ] Symbolic Expr Socket
- [ ] Nodes should be able to dynamically define new symbols on their Expr sockets.
- [ ] Expr's `FlowKind`s should be expanded:
- [ ] `Capabilities`: Expand to include subsocket checking, where Expr is the supersocket of ex. most/all of the physical, numerical, vector sockets.
- [ ] `Value`: Just the raw sympy expression, when `active_kind` is `Value`.
- [ ] `Array`: The evaluated `LazyValueFunc`, when `active_kind` is `Array`.
- Should require that the expression as a whole simplifies to `sp.Matrix`.
- Should require that there are no symbols to be defined in a socket (since `LazyValueFunc` must be called with no args).
- [ ] `LazyValueFunc`: Create a 'jax' function from an expression, such that each symbol becomes an argument to that function.
- When `active_kind` is `Value`, it should take arrays/scalars and return a scalar (expression output is a normal sympy number of some kind).
- When `active_kind` is `Array`, it should take arrays/scalars and return an array (expression output is `sp.Matrix`).
- This kind of approach allows using
- [ ] `LazyValueRange`: Expose two expressions, start/end, but with one symbol set.
- [ ] `Info`: Should always produce an `InfoFlow` that, at minimum, has an empty `dim_*`, an `output_shape` of `None`, etc., for a scalar.
- [ ] Implement an Expr Constant node to see all this through in prototype.
- [ ] Expr: Obviously, input and output.
- [ ] Symbols: Node-bound dynamic thing where you can add and subtract symbols, as well as set their type. They should popup in the `Let:` statement of the input expr socket.
- [ ] Examples: Each symbol should have the ability to set "example values", which causes the Node to fill `Params`. When all
## Keyed Cache
- [x] Implement `bl_cache.KeyedCache` for, especially, abstracting the caches underlying the input and output sockets.
- [ ] Report reason for no-link using `self.report`.
- [ ] Dropping a link on empty space should query a menu of possible nodes, or if only one node is reasonable, make that node.
## BLField as Property Abstraction
We need Python properties to work together with Blender properties.
- Blender Pros: Altered via UI. Options control UI usage. `update()` is a perfect inflection point for callback logic.
- Blender Cons: Extremely limited supported types. A lot of manual labor that duplicates work done elsewhere in a Python program
`BLField` seeks to bridge the two worlds in an elegant way.
### Type Support
We need support for arbitrary objects, but still backed by the persistance semantics of native Blender properties.
- [x] Add logic that matches appropriate types to native IntProperty, FloatProperty, IntVectorProperty, FloatVectorProperty.
- We want absolute minimal overhead for types that actually already do work in Blender.
- **REMEMBER8* they can do matrices too! https://developer.blender.org/docs/release_notes/3.0/python_api/#other-additions
- [x] Add logic that matches any bpy.types.ID subclass to a PointerProperty.
- This is important for certain kinds of properties ex. "select a Blender object".
- [ ] Implement Enum property, (also see <https://developer.blender.org/docs/release_notes/4.1/python_api/#enum-id-properties>)
- Use this to bridge the enum UI to actual StrEnum objects.
- This also maybe enables some very interesting use cases when it comes to ex. static verifiability of data provided to event callbacks.
- [x] Ensure certain options, namely `name` (as `ui_name`), `default`, `subtype`, (numeric) `min`, `max`, `step`, `precision`, (string) `maxlen`, `search`, and `search_options`, can be passed down via the `BLField()` constructor.
- [ ] Make a class method that parses the docstring.
- [ ] `description`: Use the docstring parser to extract the first description sentence of the attribute name from the subclass docstring, so we are both encouraged to document our nodes/sockets, and so we're not documenting twice.
### Niceness
- [x] Rename the internal property to 'blfield__'.
- [x] Add a method that extracts the internal property name, for places where we need the Blender property name.
- **Key use case**: `draw.prop(self, self.field_name._bl_prop_name)`, which is also nice b/c no implicit string-based reference.
- The work done above with types makes this as fast and useful as internal props. Just make sure we validate that the type can be usefully accessed like this.
- [x] Add a field method (called w/instance) that updates min/max/etc. on the 'blfield__' prop, in a native property type compatible manner: https://developer.blender.org/docs/release_notes/3.0/python_api/#idproperty-ui-data-api
- Should also throw appropriate errors for invalid access from Python, while Blender handles access from the inside.
- This allows us
- [x] Similarly, a field method that gets the 'blfield__' prop data as a dictionary.
### Parallel Features
- [x] Move serialization work to a `utils`.
- [x] Also make ENCODER a function that can shortcut the easy cases.
- [x] For serializeability, let the encoder/decoder be able to make use of an optional `.msgspec_encodable()` and similar decoder respectively, and add support for these in the ENCODER/DECODER functions.
- [x] Define a superclass for `SocketDef` and make everyone inherit from it
- [ ] Collect with a `BL_SOCKET_DEFS` object, instead of manually from `__init__.py`s
- [x] Add support for `.msgspec_*()` methods, so that we remove the dependency on sockets from the serialization module.
### Sweeping Features
- [ ] Replace all raw Blender properties with `BLField`.
- Benefit: update= is taken care of automatically, preventing an entire class of nasty bug.
- Benefit: Any serializable object can be "simply used", at almost native speed (due to the aggressive read-cache).
- Benefit: Better error properties for updating, access, setting, etc. .
- Benefit: Validate usage in a vastly greater amount of contexts.
# Overnight Ideas
- [ ] Fix file-load hiccups by persisting `_enum_cb_cache` and `_str_cb_cache`.
- [x] Implement `FlowSignal`s as special return values for `@computes_output_socket`, instead of juggling `None`.
- `FlowSignal.FlowPending`: Data was asked for, and it's not yet available, but it's expected to become available.
- Semantically: "Just hold on for a hot second".
- Return: If in any socket data provided to cb, return the same signal insted of running the callback.
- Caches: Don't invalidate caches, since the user will expect their data to still persist.
- Net Effect: Absolutely nothing happens. Perhaps we can recolor the nodes, though.
- [ ] `FlowSignal.FlowLost`: Output socket requires data that simply isn't available.
- Generally, nodes don't return it
- Return: If in any socket data provided to cb, return the same signal insted of running the callback.
- Caches: Do invalidate caches, since the user will expect their data to still persist.
- Net Effect: Sometimes, stuff happens in the output method [BB
- Net Effect: `DataChanged` is an event that signifies Node data will reset along the flow.
- [ ] Packing Imported Data in `Tidy3D Web Importer`, `Tidy3D File Importer`.
- Just `.to_hdf5_gz()` it into a `BytesIO`, Base85
- [ ] Remove Matplotlib Bottlenecks (~70ms -> ~5ms)
- Reuse `fig` per-`ManagedBLImage` (~25ms)
- Use `Agg` backend, plot with `fig.canvas.draw()`, and load image buffer directly as np.frombuffer(ax.figure.canvas.tostring_rgb(), dtype=np.uint8) (~40ms).
- [ ] Shader visualizations approximated from medium `nk` into a shader node graph, aka. a generic BSDF.

77
src/blender_maxwell/node_trees/maxwell_sim_nodes/contracts/node_types.py
View File

@ -8,8 +8,8 @@ class NodeType(blender_type_enum.BlenderTypeEnum):
# KitchenSink = enum.auto()
# Analysis
Viz = enum.auto()
ExtractData = enum.auto()
Viz = enum.auto()
## Analysis / Math
MapMath = enum.auto()
FilterMath = enum.auto()
@ -19,20 +19,20 @@ class NodeType(blender_type_enum.BlenderTypeEnum):
# Inputs
WaveConstant = enum.auto()
UnitSystem = enum.auto()
## Inputs / Scene
# Time = enum.auto()
Scene = enum.auto()
## Inputs / Constants
ExprConstant = enum.auto()
PhysicalConstant = enum.auto()
NumberConstant = enum.auto()
VectorConstant = enum.auto()
ScientificConstant = enum.auto()
UnitSystemConstant = enum.auto()
BlenderConstant = enum.auto()
## Inputs / Web Importers
Tidy3DWebImporter = enum.auto()
## Inputs / File Importers
DataFileImporter = enum.auto()
Tidy3DFileImporter = enum.auto()
## Inputs / Constants
ExprConstant = enum.auto()
ScientificConstant = enum.auto()
NumberConstant = enum.auto()
PhysicalConstant = enum.auto()
BlenderConstant = enum.auto()
# Outputs
Viewer = enum.auto()
@ -42,47 +42,49 @@ class NodeType(blender_type_enum.BlenderTypeEnum):
JSONFileExporter = enum.auto()
# Sources
## Sources / Temporal Shapes
GaussianPulseTemporalShape = enum.auto()
ContinuousWaveTemporalShape = enum.auto()
SymbolicTemporalShape = enum.auto()
DataTemporalShape = enum.auto()
## Sources /
PointDipoleSource = enum.auto()
PlaneWaveSource = enum.auto()
UniformCurrentSource = enum.auto()
# ModeSource = enum.auto()
# GaussianBeamSource = enum.auto()
# AstigmaticGaussianBeamSource = enum.auto()
# TFSFSource = enum.auto()
# EHEquivalenceSource = enum.auto()
# EHSource = enum.auto()
## Sources / Temporal Shapes
GaussianPulseTemporalShape = enum.auto()
# ContinuousWaveTemporalShape = enum.auto()
# ArrayTemporalShape = enum.auto()
TFSFSource = enum.auto()
GaussianBeamSource = enum.auto()
AstigmaticGaussianBeamSource = enum.auto()
EHDataSource = enum.auto()
EHEquivDataSource = enum.auto()
# Mediums
LibraryMedium = enum.auto()
# PECMedium = enum.auto()
# IsotropicMedium = enum.auto()
# AnisotropicMedium = enum.auto()
# TripleSellmeierMedium = enum.auto()
# SellmeierMedium = enum.auto()
# PoleResidueMedium = enum.auto()
# DrudeMedium = enum.auto()
# DrudeLorentzMedium = enum.auto()
# DebyeMedium = enum.auto()
DataFitMedium = enum.auto()
PECMedium = enum.auto()
IsotropicMedium = enum.auto()
AnisotropicMedium = enum.auto()
TripleSellmeierMedium = enum.auto()
SellmeierMedium = enum.auto()
PoleResidueMedium = enum.auto()
DrudeMedium = enum.auto()
DrudeLorentzMedium = enum.auto()
DebyeMedium = enum.auto()
## Mediums / Non-Linearities
# AddNonLinearity = enum.auto()
# ChiThreeSusceptibilityNonLinearity = enum.auto()
# TwoPhotonAbsorptionNonLinearity = enum.auto()
# KerrNonLinearity = enum.auto()
AddNonLinearity = enum.auto()
ChiThreeSusceptibilityNonLinearity = enum.auto()
TwoPhotonAbsorptionNonLinearity = enum.auto()
KerrNonLinearity = enum.auto()
# Structures
# ObjectStructure = enum.auto()
BLObjectStructure = enum.auto()
GeoNodesStructure = enum.auto()
# ScriptedStructure = enum.auto()
## Structures / Primitives
LineStructure = enum.auto()
PlaneStructure = enum.auto()
BoxStructure = enum.auto()
SphereStructure = enum.auto()
CylinderStructure = enum.auto()
PolySlabStructure = enum.auto()
# Bounds
BoundConds = enum.auto()
@ -91,7 +93,6 @@ class NodeType(blender_type_enum.BlenderTypeEnum):
PECBoundCond = enum.auto()
PMCBoundCond = enum.auto()
BlochBoundCond = enum.auto()
PeriodicBoundCond = enum.auto()
AbsorbingBoundCond = enum.auto()
# Monitors

4
src/blender_maxwell/node_trees/maxwell_sim_nodes/nodes/inputs/unit_system.py
View File

@ -3,7 +3,7 @@ from ... import sockets
from .. import base, events
class PhysicalUnitSystemNode(base.MaxwellSimNode):
class UnitSystemConstantNode(base.MaxwellSimNode):
node_type = ct.NodeType.UnitSystem
bl_label = 'Unit System'
@ -31,6 +31,6 @@ class PhysicalUnitSystemNode(base.MaxwellSimNode):
# - Blender Registration
####################
BL_REGISTER = [
PhysicalUnitSystemNode,
UnitSystemConstantNode,
]
BL_NODES = {ct.NodeType.UnitSystem: (ct.NodeCategory.MAXWELLSIM_INPUTS)}

14
src/blender_maxwell/node_trees/maxwell_sim_nodes/sockets/base.py
View File

@ -286,8 +286,7 @@ class MaxwellSimSocket(bpy.types.NodeSocket):
"""
self.display_shape = (
'SQUARE'
if self.active_kind
in {ct.FlowKind.LazyValueFunc, ct.FlowKind.LazyValueRange}
if self.active_kind in {ct.FlowKind.Array, ct.FlowKind.LazyValueRange}
else 'CIRCLE'
) + ('_DOT' if self.use_units else '')
@ -882,7 +881,6 @@ class MaxwellSimSocket(bpy.types.NodeSocket):
{
ct.FlowKind.Value: self.draw_value,
ct.FlowKind.Array: self.draw_array,
ct.FlowKind.LazyValueFunc: self.draw_lazy_value_func,
ct.FlowKind.LazyArrayRange: self.draw_lazy_array_range,
}[self.active_kind](col)
@ -1000,16 +998,6 @@ class MaxwellSimSocket(bpy.types.NodeSocket):
col: Target for defining UI elements.
"""
def draw_lazy_value_func(self, col: bpy.types.UILayout) -> None:
"""Draws the socket lazy value on its own line.
Notes:
Should be overriden by individual socket classes, if they have an editable `FlowKind.LazyValueFunc`.
Parameters:
col: Target for defining UI elements.
"""
def draw_lazy_array_range(self, col: bpy.types.UILayout) -> None:
"""Draws the socket lazy array range on its own line.

4
src/blender_maxwell/node_trees/maxwell_sim_nodes/sockets/basic/expr.py
View File

@ -69,15 +69,13 @@ class ExprBLSocket(base.MaxwellSimSocket):
####################
@property
def value(self) -> sp.Expr:
expr = sp.sympify(
return sp.sympify(
self.raw_value,
locals={sym.name: sym for sym in self.symbols},
strict=False,
convert_xor=True,
).subs(spux.ALL_UNIT_SYMBOLS)
return expr
@value.setter
def value(self, value: str) -> None:
self.raw_value = sp.sstr(value)