diff --git a/TODO.md b/TODO.md index 854a0f1..0c5625e 100644 --- a/TODO.md +++ b/TODO.md @@ -1,93 +1,105 @@ -# Acute Tasks -- [x] Implement Material Import for Maxim Data -- [x] Implement Robust DataFlowKind for list-like / spectral-like composite types -- [x] Unify random node/socket caches. -- [x] Revalidate cache logic -- [x] Finish math system -- [ ] Finish the "Low-Hanging Fruit" Nodes -- [ ] Move preview GN trees to the asset library. +# Working TODO +- [ ] Wave Constant +- [ ] Fix many problems by persisting `_enum_cb_cache` and `_str_cb_cache`. + # Nodes ## Analysis - [x] Extract + - [ ] Implement "saved" state for info, and provide the user an indicator that state has been saved w/a button to reset (the state should also be reset when plugging a new data thing in) - [x] Viz + - [ ] Implement Info-driven planar projection of pixels onto managed image empty. + - [ ] Live-slice 2D field values onto user-controlled image empty from 2D field. + - [ ] SocketType-based visualization support. + - [ ] Pol SocketType: 2D elliptical visualization of Jones vectors. + - [ ] Pol SocketType: 3D Poincare sphere visualization of Stokes vectors. + - [x] Math / Map Math - [ ] Remove "By x" socket set let socket sets only be "Function"/"Expr"; then add a dynamic enum underneath to select "By x" based on data support. - [ ] Filter the operations based on data support, ex. use positive-definiteness to guide cholesky. + - [ ] Implement support for additional symbols via `Expr`. - [x] Math / Filter Math - [ ] Math / Reduce Math -- [ ] Math / Operate Math +- [x] Math / Operate Math + - [ ] Remove two-layered dropdown; directly filter operations and use categories to seperate them. + - [ ] Implement Expr socket advancements to make a better experience operating between random expression-like sockets. ## Inputs - [x] Wave Constant - - [x] Implement export of frequency / wavelength array/range. -- [x] Unit System - - [ ] Implement presets, including "Tidy3D" and "Blender", shown in the label row. + - [ ] Fix the LazyValueRange (again!) + - [ ] Document +- [x] Scene + - [ ] Implement export of scene time via. Blender unit system. + - [ ] Implement optional scene-synced time exporting, so that the simulation definition and scene definition match for analysis needs. -- [x] Constants / Scientific Constant - - [x] Create `utils.sci_constants` to map `scipy` constants to `sympy` units. - - [x] Utilize `utils.sci_constants` to make it easy for the user to select appropriate constants with two-layered dropdowns. +- [x] Constants / Expr Constant + - See IDEAS. - [x] Constants / Number Constant + - [ ] Fix non-integer sockets +- [ ] Constants / Vector Constant - [ ] Constants / Physical Constant - - [ ] Pol: Elliptical viz as 2D plot. - - [ ] Pol: Poincare sphere viz as 3D GN. +- [x] Constants / Scientific Constant + - [ ] Nicer (boxed?) node information, maybe centered headers, in a box, etc. . +- [x] Constants / Unit System Constant + - [ ] Implement presets, including "Tidy3D" and "Blender", shown in the label row. - [x] Constants / Blender Constant + - [ ] Fix it! - [ ] Web / Tidy3D Web Importer - - [ ] Have a visual indicator for the current download status, with a manual re-download button. + - [ ] Fix the check of folders, actually, just fix `tdcloud` in general! + - [ ] Have a visual indicator for the download status of the currently selected task, as well as its data size. + - [ ] If a task is "selected", lock the cloud task socket, so other tasks can't be selected. While that lock is active, expose a real "download" button. Also make the loose output socket and put out a `FlowPending` until the download is available. + - [ ] A manual download button and seperate re-download button (maybe on the side, round reload boi). + - [ ] An option to pack the data into the blend, with overview of how much data it will take (Base85/base64 has overhead). + - [ ] Default limits for caching/packing. + - [ ] Support importing batched simulations and outputting an `Array` of SimData. -- [x] File Import / Material Import - - [x] Dropdown to choose import format - - MERGED w/TIDY3D FILE IMPORT -- [x] File Import / Tidy3D File Import - - [x] HDF and JSON file support, with appropriate choice of loose output socket. -- [ ] File Import / Array File Import +- [ ] File Import / Data File Import + - [ ] Implement `FlowKind.LazyValueFunc` that plays the loading game. + - [ ] Implement `FlowKind.Info` which lets the user describe the data being loaded, for proper further processing. + - [ ] Implement unit system input to guide conversion from numpy data type. + - [ ] Implement datatype dropdown to guide format from disk, prefilled to detected. + - [ ] Implement `FlowKind.Array` that just runs the `LazyValueFunc` as usual. - [ ] Standardize 1D and 2D array loading/saving on numpy's savetxt with gzip enabled. - - [ ] Implement unit system input to guide conversion from numpy data type. - - [ ] Implement datatype dropdown to guide format from disk, prefilled to detected. - - [ ] Implement a LazyValue to provide a data path that avoids having to load massive arrays every time always. +- [x] File Import / Tidy3D File Import ## Outputs - [x] Viewer - - [ ] Remove image preview when disabling plots. - - [x] Auto-enable 3D preview when creating. + - [ ] Consider a "debug" mode + - [ ] Auto-enable plot when creating. - [ ] Test/support multiple viewers at the same time. - [ ] Pop-up w/multiline string as alternative to console print. + - [ ] Handle per-tree viewers, so that switching trees doesn't "bleed" state from the old tree. + - [ ] BUG: CTRL+SHIFT+CLICK not on a node shows an error; should just do nothing. - [x] Web Export / Tidy3D Web Exporter - - This is an extraordinarily nuanced node, and will need constant adjusting if it is to be robust. + - [ ] Run checks on-demand, and require they be run before the sim can be uploaded. If the simulation changes, don't + - [ ] Support doing checks in a seperate process. - [ ] We need better ways of doing checks before uploading, like for monitor data size. Maybe a SimInfo node? - - [ ] Implement "new folder" feature w/popup operator. - - [ ] Implement "delete task" feature w/popup confirmation. - - [ ] We need to be able to "delete and re-upload" (or maybe just delete from the interface). + - [ ] Accept `Array` of simulations, and upload them as `Batch`. - [x] File Export / JSON File Export -- [ ] File Import / Tidy3D File Export + - [ ] Reevaluate its purpose. +- [ ] File Export / Tidy3D File Export - [ ] Implement HDF-based export of Tidy3D-exported object (which includes ex. mesh data and such) - [ ] Also JSON (but indicate somehow that ex. mesh data doesn't come along for the ride). -- [ ] File Export / Array File Export +- [ ] File Export / Data File Export - [ ] Implement datatype dropdown to guide format on disk. - [ ] Implement unit system input to guide conversion to numpy data type. - [ ] Standardize 1D and 2D array loading/saving on numpy's savetxt with gzip enabled. -## Viz -- [x] Monitor Data Viz - - [x] Implement dropdown to choose which monitor in the SimulationData should be visualized (based on which are available in the SimulationData), and implement visualization based on every kind of monitor-adjascent output data type () - - [ ] Project field values onto a plane object (managed) - ## Sources - [x] Temporal Shapes / Gaussian Pulse Temporal Shape - [x] Temporal Shapes / Continuous Wave Temporal Shape - [ ] Temporal Shapes / Symbolic Temporal Shape - [ ] Specify a Sympy function to generate appropriate array based on -- [ ] Temporal Shapes / Array Temporal Shape +- [ ] Temporal Shapes / Data Temporal Shape - [x] Point Dipole Source - [ ] Use a viz mesh, not empty (empty doesn't play well with alpha hashing). - [ ] Plane Wave Source - - [x] Implement an oriented vector input with 3D preview. - [ ] **IMPORTANT**: Fix the math so that an actually valid construction emerges!! - [ ] Uniform Current Source - [ ] TFSF Source @@ -95,14 +107,17 @@ - [ ] Gaussian Beam Source - [ ] Astigmatic Gaussian Beam Source -- [ ] Mode Source - -- [ ] Array Source / EH Array Source -- [ ] Array Source / EH Equiv Array Source +- [ ] EH Array Source +- [ ] EH Equiv Array Source ## Mediums - [x] Library Medium - - [ ] Implement frequency range output (listy) + - [ ] Implement frequency range output (listy), perhaps in the `InfoFlow` lane? + - [ ] Implement dynamic label. +- [ ] DataFit Medium + - [ ] Implement by migrating the material data fitting logic from the `Tidy3D File Importer`, except now only accept a `Data` input socket, and rely on the `Data File Importer` to do the parsing into an acceptable `Data` socket format. + - [ ] Save the result in the node, specifically in a property (serialized!) and lock the input graph while saved. + - [ ] PEC Medium - [ ] Isotropic Medium - [ ] Anisotropic Medium @@ -117,55 +132,48 @@ - [ ] Non-Linearity / Two-Photon Absorption Non-Linearity - [ ] Non-Linearity / Kerr Non-Linearity -- [ ] Space/Time epsilon/mu Modulation - ## Structures - [ ] BLObject Structure - [x] GeoNodes Structure - - [x] Rewrite the `bl_socket_map.py` - - [x] Use the modifier itself as memory, via the ManagedObj - - [x] Rewrite to use unit systems properly. + - [ ] Implement a panel system, to make GN trees with a ton of inputs (most of which are not usually needed) actually useful. - [ ] Propertly map / implement Enum input sockets to the GN group. - - [ ] Implement a panel system, either based on native GN panels, or description parsing, or something like that. +- [ ] Primitive Structures / Line Structure - [ ] Primitive Structures / Plane Structure - [x] Primitive Structures / Box Structure - [x] Primitive Structures / Sphere Structure - [ ] Primitive Structures / Cylinder Structure -- [ ] Primitive Structures / Ring Structure -- [ ] Primitive Structures / Capsule Structure -- [ ] Primitive Structures / Cone Structure - -## Monitors -- [x] E/H Field Monitor -- [x] Field Power Flux Monitor -- [ ] \epsilon Tensor Monitor -- [ ] Diffraction Monitor -- [ ] Axis-aligned planar 2D (pixel) - -- [ ] Projected E/H Field Monitor / Cartesian Projected E/H Field Monitor - - [ ] Use to implement the metalens: -- [ ] Projected E/H Field Monitor / Angle Projected E/H Field Monitor -- [ ] Projected E/H Field Monitor / K-Space Projected E/H Field Monitor - -- [ ] Modal Nodes - - Spatial+frequency feature monitoring. An EM field can be decomposed into using a specially configured solver, which can be used to look for very particular kinds of effects by constraining investigations of a solver result to filter out everything that isn't these particular modes aka. features. Kind of a fourier-based redimensionalization, almost). - -## Simulations -- [x] FDTDSim - -- [x] Sim Domain - - [ ] By-Medium batching of Structures when building the td.Simulation object, which can have significant performance implications. +- [ ] Primitive Structures / PolySlab Structure +## Bounds - [x] Boundary Conds - [ ] Boundary Cond / PML Bound Face - [ ] Dropdown for "Normal" and "Stable" - [ ] Boundary Cond / PEC Bound Face - [ ] Boundary Cond / PMC Bound Face - [ ] Boundary Cond / Bloch Bound Face -- [ ] Boundary Cond / Periodic Bound Face + - [ ] Implement "simple" mode aka "periodic" mode in Tidy3D - [ ] Boundary Cond / Absorbing Bound Face +## Monitors +- [x] EH Field Monitor + - [ ] Method of setting `inf` on dimensions - use a `ManyEnum` maybe to select the injection axis, and let that set the $0$. + - [ ] Revamp the input parameters. +- [x] Power Flux Monitor +- [ ] Permittivity Monitor +- [ ] Diffraction Monitor + +- [ ] Projected E/H Field Monitor / Cartesian Projected E/H Field Monitor + - [ ] Use to implement the metalens: +- [ ] Projected E/H Field Monitor / Angle Projected E/H Field Monitor +- [ ] Projected E/H Field Monitor / K-Space Projected E/H Field Monitor + +## Simulations +- [x] FDTDSim + - [ ] By-Medium batching of Structures when building the td.Simulation object, which can have significant performance implications. + +- [x] Sim Domain + - [ ] Sim Grid - [ ] Sim Grid Axes / Auto Sim Grid Axis - [ ] Sim Grid Axes / Manual Sim Grid Axis @@ -174,8 +182,10 @@ ## Utilities - [ ] Separate + - [ ] Use generic Expr socket mode to combine numerical types into either Expr or Data socket. - [x] Combine - - [x] Implement concatenation of sim-critical socket types into their multi-type + - [ ] Use generic Expr socket mode to combine numerical types into either Expr or Data socket. + - [ ] Explicit about lower structures taking precedence. @@ -183,34 +193,30 @@ - [ ] Tests / Monkey (suzanne deserves to be simulated, she may need manifolding up though :)) - [ ] Tests / Wood Pile +- [ ] Structures / Primitives / Line - [ ] Structures / Primitives / Plane - [x] Structures / Primitives / Box - [x] Structures / Primitives / Sphere - [ ] Structures / Primitives / Cylinder - [x] Structures / Primitives / Ring -- [ ] Structures / Primitives / Capsule -- [ ] Structures / Primitives / Cone -- [ ] Structures / Arrays / Square -- [ ] Structures / Arrays / Square-Hole - [ ] Structures / Arrays / Cyl -- [ ] Structures / Arrays / Cyl-Hole -- [x] Structures / Arrays / Box -- [x] Structures / Arrays / Sphere +- [ ] Structures / Arrays / Box +- [ ] Structures / Arrays / Sphere - [ ] Structures / Arrays / Cylinder - [x] Structures / Arrays / Ring -- [ ] Structures / Arrays / Capsule -- [ ] Structures / Arrays / Cone -- [ ] Array / Square Array **NOTE: Ring and cylinder** -- [ ] Array / Hex Array **NOTE: Ring and cylinder** -- [ ] Hole Array / Square Hole Array: Takes a primitive hole shape. -- [ ] Hole Array / Hex Hole Array: Takes a primitive hole shape. -- [ ] Cavity Array / Hex Array w/ L-Cavity -- [ ] Cavity Array / Hex Array w/ H-Cavity +- [ ] Structures / Hex Arrays / Cyl +- [ ] Structures / Hex Arrays / Box +- [ ] Structures / Hex Arrays / Sphere +- [ ] Structures / Hex Arrays / Cylinder +- [x] Structures / Hex Arrays / Ring -- [ ] Crystal Sphere Lattice / Sphere FCC Array -- [ ] Crystal Sphere Lattice / Sphere BCC Array +- [ ] Structures / Cavity Arrays / L-Cavity Cylinder +- [ ] Structures / Cavity Arrays / H-Cavity Cylinder + +- [ ] Structures / Lattice Arrays / FCC Sphere +- [ ] Structures / Lattice Arrays / BCC Sphere @@ -233,6 +239,9 @@ - [x] String - [x] File Path - [x] Color +- [x] Expr + - [ ] Implement node-driven support for dynamic symbols. + - [ ] Implement compatibility with sockets that fundamentally do produce expressions, especially Physical sockets. ## Number - [x] Integer @@ -279,7 +288,10 @@ ## Tidy3D - [x] Cloud Task - - [ ] Implement switcher for API-key-having config filconfig file vs. direct entry of API key. It should be auto-filled with the config file when such a thing exists. + - [ ] Move API checking out of the socket, and don't re-prompt for a key if the config file exists. + - [ ] Remove the existing task selector when making a new task. + - [ ] Implement "new folder" feature w/popup operator. + - [ ] Implement "delete task" feature w/popup confirmation. ## Physical - [x] Unit System @@ -350,16 +362,18 @@ # Internal / Architecture -## IDEAS -- [ ] Socket "guarding" - let nodes influence the dynamic capabilities of sockets to prevent links (with a `self.report` explanation) to an output socket that won't yet produce a value. - - [ ] Prevents some uses of loose sockets (we want less loose sockets!) - ## CRITICAL +- [ ] Rethink the way that loose sockets are replaced, specifically with respect to deterministic ordering. + - Currently order is not guaranteed. This is causing problems. + + +## User-Facing Errors and Legal Considerations - [ ] `log.error` should invoke `self.report` in some Blender operator - used for errors that are due to usage error (which can't simply be prevented with UX design, like text file formatting of import), not due to error in the program. - [ ] License header UI for MaxwellSimTrees, to clarify the AGPL-compatible potentially user-selected license that trees must be distributed under. -- [x] Document the node tree cache semantics thoroughly; it's a VERY nuanced piece of logic, and its invariants may not survive Blender versions / the author's working memory -- [ ] Start standardizing nodes/sockets w/individualized SemVer +- [ ] A "CitationsFlow" FlowKind which simply propagates citations. +- [ ] Implement standardization of nodes/sockets w/individualized SemVer - Perhaps keep node / socket versions in a property, so that trying to load an incompatible major version hop can error w/indicator of where to find a compatible `blender_maxwell` version. + - Integrate w/BLField, to help the user manage addon updates that would break their tree. ## Documentation - [ ] Make all modules available @@ -368,16 +382,12 @@ - [ ] Comb through and finish `__doc__`s. ## Performance -- [ ] The GN value pushing currently does an expensive from-sympy conversion for all GN attributes, making it very slow. - - Generally, the issue is that we can't compare the pushed value to the existing value without a conversion. - - Also, sympy expressions can't be hashed (by default), and `str()` may be just as slow, so a simple `@lru_cache` is no good. - - **One approach** is a `HashableSympyExpr` class, which would use ex. atoms and values in `SympyExpr` to let us `@lru_cache` the sympy conversion. - - **Another approach** is to "just" make the `scale_to_unit` function faster (we should profile). - - Presumably, the scaling factors between units could be eagerly cached, then the numerical part of the expression could be used to avoid `convert_to` calls. - - Without the list format of `(np.array, spu.Quantity)`, `sp.Matrix` can't be performantly handled like this. +- [ ] Optimize GN value pushing w/sympy expression hashing. ## Style -- [ ] Color of nodes should be a nice blue, eh? +Header color style can't be done, unfortunately. Body color feels unclean, so nothing there for now. + +- [ ] Node icons to denote preview/plot state. ## Registration and Contracts - [ ] Refactor the node category code; it's ugly. @@ -396,47 +406,30 @@ - [ ] Document the `tdcloud` service thoroughly and open a GitHub discussion about `td.web` shortcomings. ## Node Base Class -- [ ] Implement a new socket type for preview-only parameters - - [ ] When used, the node base class should expose a toggle - - [ ] Instead of mangling props, we can instead reuse all of the socket-based code, while also enabling composability of previews. -- [ ] Custom `@cache`/`@lru_cache`/`@cached_property` which caches by instance ID (possibly based on `beartype` or `pydantic`). - - The problem to solve is performance vs. persistence. -- [ ] -- [ ] Implement by-category sorting of loose sockets with 'move' method on `node.inputs`/`node.outputs`. - - Currently order is not guaranteed -- [ ] When presets are used, if a preset is selected and the user alters a preset setting, then dynamically switch the preset indicator back to "Custom" to indicate that there is no active preset +- [ ] Re-engineer "presets" to use an Enum of some kind. ## Events -- [x] Mechanism for selecting a blender object managed by a particular node. - - [ ] Standard way of triggering the selection -- [ ] Mechanism for ex. specially coloring a node that is currently participating in the preview. -- [ ] Custom callbacks when deleting a node (in `free()`), to ex. delete all previews with the viewer node. +- [ ] When a Blender object is selected, select the node that owns its ManagedObj. +- [ ] Node button / shortcut / something to select the ManagedObj owned by a node. ## Socket Base Class -- [ ] Second-generation listy, based on a `DataFlowKind.ValueListy`, `DataFlowKind.ValueRange`, etc. to encode the presence of special logic. - - This is key to allow special handling, as "just give me a `list[]` of `sympy` objects" is an exceptionally non-performant and brittle thing. -- [ ] Implement capability system, which defaults to exactly matching the type. - - [ ] Make `to_socket`s no-consent to new links from `from_socket`s of incompatible Capability. - - [ ] Add Capabilities needed mainly in cases where we need `Any` currently. +- [ ] Collect `SocketDef` objects like we do with `BL_REGISTER`, without any special mojo sauce. ## Many Nodes - [ ] Implement "Steady-State" / "Time Domain" on all relevant Monitor nodes -- [ ] (?) Dynamic `bl_label` where appropriate (ex. "Library Medium" becoming "Au Medium") -- [ ] Implement LazyValue, including LazyParamValue on a new class of constant-like input nodes that really just emit ex. sympy variables. - [ ] Medium Features - [ ] Accept spatial field. Else, spatial uniformity. - [ ] Accept non-linearity. Else, linear. - [ ] Accept space-time modulation. Else, static. - [ ] Modal Features - - [ ] ModeSpec, for use by ModeSource, ModeMonitor, ModeSolverMonitor. Data includes ModeSolverData, ModeData, ScalarModeFieldDataArray, ModeAmpsDataArray, ModeIndexDataArray, ModeSolver. + - ModeSpec, for use by ModeSource, ModeMonitor, ModeSolverMonitor. Data includes ModeSolverData, ModeData, ScalarModeFieldDataArray, ModeAmpsDataArray, ModeIndexDataArray, ModeSolver. ## Many Sockets -- [ ] Implement constrained SympyExpr checks all over the place. ## Development Tooling +- [ ] Implement `pre-commit. - [ ] Pass a `mypy` check - [ ] Pass all `ruff` checks, including `__doc__` availability. -- [ ] Implement `pre-commit. - [ ] Add profiling support, so we can properly analyze performance characteristics. - Without a test harness, or profile-while-logging, there may be undue noise in our analysis. - [ ] Simple `pytest` harnesses for unit testing of nodes, sockets. @@ -454,41 +447,24 @@ - We might still want/need the jax based stuff after; volume geonodes aren't finalized. ## Packaging -- [ ] Popup to install dependencies after UI is available (possibly with the help of the `draw()` function of the `InstallPyDeps` operator) -- [ ] Use a Modal and multiline-text-like construction to print `pip install` as we install dependencies, so that the user has an idea that something is happening. - [ ] Test lockfile platform-agnosticism on Windows +## BLCache +- [ ] Replace every raw property with `BLField`. +- [ ] Add matrix property support: https://developer.blender.org/docs/release_notes/3.0/python_api/#other-additions +- [ ] Fix many problems by persisting `_enum_cb_cache` and `_str_cb_cache`. +- [ ] Docstring parser for descriptions. +- [ ] Method of dynamically setting property options after creation, using `idproperty_ui_data` + # BUGS We're trying to do our part by reporting bugs we find! -This is where we keep track of them for now. +This is where we keep track of them for now, if they're not covered by the above listings. ## Blender Maxwell Bugs -- [ ] Detaching data chained into Viz node makes for a very laggy error, as non-implemented LazyValueFunc suddenly can't propagate live into the Viz node. -- [ ] Need to clear invalid searched StrProperties on copy -- [ ] Enabled 3D preview is really slow for some reason when working with the math nodes. - -- [ ] BUG: CTRL+SHIFT+CLICK not on a node shows an error; should just do nothing. -- [ ] Slow changing of socket sets / range on wave constant. -- [ ] API auth shouldn't show if everything is fine in Cloud Task socket -- [ ] Cloud task socket loads folders before its node shows, which can be slow (and error prone if offline) -- [ ] Dispersive fit is slow, which means lag on normal operations that rely on the fit result - fit computation should be integrated into the node, and the output socket should only appear when the fit is available. -- [ ] Numerical, Physical Constant is missing entries -- [ ] Numerical, Physical Constant is missing entries - -BROKE NODES -- [ ] Numerical constant doesn't switch types -- [ ] Blender constant is inexplicably mega laggy -- [ ] Web importer is just wonky in general -- [ ] JSON File exporter is having trouble with generic types (is that bad?) - -- [ ] Extact Data needs flux settings -- [ ] Point dipole still has no preview -- [ ] Plane wave math still doesn't work and it has no preview -- [ ] Monitors need a way of setting infinite dimensions ## Blender Bugs Reported: @@ -511,87 +487,39 @@ Unreported: - Let's have operator `poll_message_set`: https://projects.blender.org/blender/blender/commit/ebe04bd3cafaa1f88bd51eee5b3e7bef38ae69bc - Careful, Python uses user site packages: - Our modifier obj can see execution time: -- We found the translation callback! https://projects.blender.org/blender/blender/commit/8564e03cdf59fb2a71d545e81871411b82f561d9 - - This can update the node center!! -- [x] Optimize the `DataChanged` invalidator. -- [ ] Optimize unit stripping. +## IDEAS +- [ ] Depedencies-gated addon preferences. + - [ ] Preferences-based specification/alteration of default unit systems for Tidy3D and Blender. + - [ ] Preferences-based specification/alteration of Tidy3D API key, so we can factor away all the `prelock` bullshit. +- [ ] Subsockets + - We need Exprs to not be so picky. + - All the sympy-making nodes should be subsockets of Expr, so that you can plug any socket that should work with Expr into Expr. + - When it comes to Data, any Expr that produces an array-like output from its `LazyValueFunc` should be deemed compatible (as in, the Expr may plug into a Data socket). + - Specifically, that means the presence of a well-defined `Info`, as well as `jax` compatibility. +- [ ] Symbolic Expr Socket + - [ ] Nodes should be able to dynamically define new symbols on their Expr sockets. + - [ ] Expr's `FlowKind`s should be expanded: + - [ ] `Capabilities`: Expand to include subsocket checking, where Expr is the supersocket of ex. most/all of the physical, numerical, vector sockets. + - [ ] `Value`: Just the raw sympy expression, when `active_kind` is `Value`. + - [ ] `Array`: The evaluated `LazyValueFunc`, when `active_kind` is `Array`. + - Should require that the expression as a whole simplifies to `sp.Matrix`. + - Should require that there are no symbols to be defined in a socket (since `LazyValueFunc` must be called with no args). + - [ ] `LazyValueFunc`: Create a 'jax' function from an expression, such that each symbol becomes an argument to that function. + - When `active_kind` is `Value`, it should take arrays/scalars and return a scalar (expression output is a normal sympy number of some kind). + - When `active_kind` is `Array`, it should take arrays/scalars and return an array (expression output is `sp.Matrix`). + - This kind of approach allows using + - [ ] `LazyValueRange`: Expose two expressions, start/end, but with one symbol set. + - [ ] `Info`: Should always produce an `InfoFlow` that, at minimum, has an empty `dim_*`, an `output_shape` of `None`, etc., for a scalar. + - [ ] Implement an Expr Constant node to see all this through in prototype. + - [ ] Expr: Obviously, input and output. + - [ ] Symbols: Node-bound dynamic thing where you can add and subtract symbols, as well as set their type. They should popup in the `Let:` statement of the input expr socket. + - [ ] Examples: Each symbol should have the ability to set "example values", which causes the Node to fill `Params`. When all -## Keyed Cache -- [x] Implement `bl_cache.KeyedCache` for, especially, abstracting the caches underlying the input and output sockets. +- [ ] Report reason for no-link using `self.report`. +- [ ] Dropping a link on empty space should query a menu of possible nodes, or if only one node is reasonable, make that node. - -## BLField as Property Abstraction -We need Python properties to work together with Blender properties. -- Blender Pros: Altered via UI. Options control UI usage. `update()` is a perfect inflection point for callback logic. -- Blender Cons: Extremely limited supported types. A lot of manual labor that duplicates work done elsewhere in a Python program - -`BLField` seeks to bridge the two worlds in an elegant way. - -### Type Support -We need support for arbitrary objects, but still backed by the persistance semantics of native Blender properties. -- [x] Add logic that matches appropriate types to native IntProperty, FloatProperty, IntVectorProperty, FloatVectorProperty. - - We want absolute minimal overhead for types that actually already do work in Blender. - - **REMEMBER8* they can do matrices too! https://developer.blender.org/docs/release_notes/3.0/python_api/#other-additions -- [x] Add logic that matches any bpy.types.ID subclass to a PointerProperty. - - This is important for certain kinds of properties ex. "select a Blender object". -- [ ] Implement Enum property, (also see ) - - Use this to bridge the enum UI to actual StrEnum objects. - - This also maybe enables some very interesting use cases when it comes to ex. static verifiability of data provided to event callbacks. -- [x] Ensure certain options, namely `name` (as `ui_name`), `default`, `subtype`, (numeric) `min`, `max`, `step`, `precision`, (string) `maxlen`, `search`, and `search_options`, can be passed down via the `BLField()` constructor. - - [ ] Make a class method that parses the docstring. - - [ ] `description`: Use the docstring parser to extract the first description sentence of the attribute name from the subclass docstring, so we are both encouraged to document our nodes/sockets, and so we're not documenting twice. - -### Niceness -- [x] Rename the internal property to 'blfield__'. -- [x] Add a method that extracts the internal property name, for places where we need the Blender property name. - - **Key use case**: `draw.prop(self, self.field_name._bl_prop_name)`, which is also nice b/c no implicit string-based reference. - - The work done above with types makes this as fast and useful as internal props. Just make sure we validate that the type can be usefully accessed like this. -- [x] Add a field method (called w/instance) that updates min/max/etc. on the 'blfield__' prop, in a native property type compatible manner: https://developer.blender.org/docs/release_notes/3.0/python_api/#idproperty-ui-data-api - - Should also throw appropriate errors for invalid access from Python, while Blender handles access from the inside. - - This allows us -- [x] Similarly, a field method that gets the 'blfield__' prop data as a dictionary. - -### Parallel Features -- [x] Move serialization work to a `utils`. -- [x] Also make ENCODER a function that can shortcut the easy cases. -- [x] For serializeability, let the encoder/decoder be able to make use of an optional `.msgspec_encodable()` and similar decoder respectively, and add support for these in the ENCODER/DECODER functions. -- [x] Define a superclass for `SocketDef` and make everyone inherit from it - - [ ] Collect with a `BL_SOCKET_DEFS` object, instead of manually from `__init__.py`s - - [x] Add support for `.msgspec_*()` methods, so that we remove the dependency on sockets from the serialization module. - -### Sweeping Features -- [ ] Replace all raw Blender properties with `BLField`. - - Benefit: update= is taken care of automatically, preventing an entire class of nasty bug. - - Benefit: Any serializable object can be "simply used", at almost native speed (due to the aggressive read-cache). - - Benefit: Better error properties for updating, access, setting, etc. . - - Benefit: Validate usage in a vastly greater amount of contexts. - - - -# Overnight Ideas -- [ ] Fix file-load hiccups by persisting `_enum_cb_cache` and `_str_cb_cache`. - -- [x] Implement `FlowSignal`s as special return values for `@computes_output_socket`, instead of juggling `None`. - - `FlowSignal.FlowPending`: Data was asked for, and it's not yet available, but it's expected to become available. - - Semantically: "Just hold on for a hot second". - - Return: If in any socket data provided to cb, return the same signal insted of running the callback. - - Caches: Don't invalidate caches, since the user will expect their data to still persist. - - Net Effect: Absolutely nothing happens. Perhaps we can recolor the nodes, though. - - - [ ] `FlowSignal.FlowLost`: Output socket requires data that simply isn't available. - - Generally, nodes don't return it - - Return: If in any socket data provided to cb, return the same signal insted of running the callback. - - Caches: Do invalidate caches, since the user will expect their data to still persist. - - Net Effect: Sometimes, stuff happens in the output method [BB - - Net Effect: `DataChanged` is an event that signifies Node data will reset along the flow. - -- [ ] Packing Imported Data in `Tidy3D Web Importer`, `Tidy3D File Importer`. - - Just `.to_hdf5_gz()` it into a `BytesIO`, Base85 - -- [ ] Remove Matplotlib Bottlenecks (~70ms -> ~5ms) - - Reuse `fig` per-`ManagedBLImage` (~25ms) - - Use `Agg` backend, plot with `fig.canvas.draw()`, and load image buffer directly as np.frombuffer(ax.figure.canvas.tostring_rgb(), dtype=np.uint8) (~40ms). +- [ ] Shader visualizations approximated from medium `nk` into a shader node graph, aka. a generic BSDF. diff --git a/src/blender_maxwell/node_trees/maxwell_sim_nodes/contracts/node_types.py b/src/blender_maxwell/node_trees/maxwell_sim_nodes/contracts/node_types.py index eafe451..17fa458 100644 --- a/src/blender_maxwell/node_trees/maxwell_sim_nodes/contracts/node_types.py +++ b/src/blender_maxwell/node_trees/maxwell_sim_nodes/contracts/node_types.py @@ -8,8 +8,8 @@ class NodeType(blender_type_enum.BlenderTypeEnum): # KitchenSink = enum.auto() # Analysis - Viz = enum.auto() ExtractData = enum.auto() + Viz = enum.auto() ## Analysis / Math MapMath = enum.auto() FilterMath = enum.auto() @@ -19,20 +19,20 @@ class NodeType(blender_type_enum.BlenderTypeEnum): # Inputs WaveConstant = enum.auto() - UnitSystem = enum.auto() - - ## Inputs / Scene - # Time = enum.auto() + Scene = enum.auto() + ## Inputs / Constants + ExprConstant = enum.auto() + PhysicalConstant = enum.auto() + NumberConstant = enum.auto() + VectorConstant = enum.auto() + ScientificConstant = enum.auto() + UnitSystemConstant = enum.auto() + BlenderConstant = enum.auto() ## Inputs / Web Importers Tidy3DWebImporter = enum.auto() ## Inputs / File Importers + DataFileImporter = enum.auto() Tidy3DFileImporter = enum.auto() - ## Inputs / Constants - ExprConstant = enum.auto() - ScientificConstant = enum.auto() - NumberConstant = enum.auto() - PhysicalConstant = enum.auto() - BlenderConstant = enum.auto() # Outputs Viewer = enum.auto() @@ -42,47 +42,49 @@ class NodeType(blender_type_enum.BlenderTypeEnum): JSONFileExporter = enum.auto() # Sources + ## Sources / Temporal Shapes + GaussianPulseTemporalShape = enum.auto() + ContinuousWaveTemporalShape = enum.auto() + SymbolicTemporalShape = enum.auto() + DataTemporalShape = enum.auto() ## Sources / PointDipoleSource = enum.auto() PlaneWaveSource = enum.auto() UniformCurrentSource = enum.auto() - # ModeSource = enum.auto() - # GaussianBeamSource = enum.auto() - # AstigmaticGaussianBeamSource = enum.auto() - # TFSFSource = enum.auto() - # EHEquivalenceSource = enum.auto() - # EHSource = enum.auto() - ## Sources / Temporal Shapes - GaussianPulseTemporalShape = enum.auto() - # ContinuousWaveTemporalShape = enum.auto() - # ArrayTemporalShape = enum.auto() + TFSFSource = enum.auto() + GaussianBeamSource = enum.auto() + AstigmaticGaussianBeamSource = enum.auto() + EHDataSource = enum.auto() + EHEquivDataSource = enum.auto() # Mediums LibraryMedium = enum.auto() - # PECMedium = enum.auto() - # IsotropicMedium = enum.auto() - # AnisotropicMedium = enum.auto() - # TripleSellmeierMedium = enum.auto() - # SellmeierMedium = enum.auto() - # PoleResidueMedium = enum.auto() - # DrudeMedium = enum.auto() - # DrudeLorentzMedium = enum.auto() - # DebyeMedium = enum.auto() - + DataFitMedium = enum.auto() + PECMedium = enum.auto() + IsotropicMedium = enum.auto() + AnisotropicMedium = enum.auto() + TripleSellmeierMedium = enum.auto() + SellmeierMedium = enum.auto() + PoleResidueMedium = enum.auto() + DrudeMedium = enum.auto() + DrudeLorentzMedium = enum.auto() + DebyeMedium = enum.auto() ## Mediums / Non-Linearities - # AddNonLinearity = enum.auto() - # ChiThreeSusceptibilityNonLinearity = enum.auto() - # TwoPhotonAbsorptionNonLinearity = enum.auto() - # KerrNonLinearity = enum.auto() + AddNonLinearity = enum.auto() + ChiThreeSusceptibilityNonLinearity = enum.auto() + TwoPhotonAbsorptionNonLinearity = enum.auto() + KerrNonLinearity = enum.auto() # Structures - # ObjectStructure = enum.auto() + BLObjectStructure = enum.auto() GeoNodesStructure = enum.auto() - # ScriptedStructure = enum.auto() ## Structures / Primitives + LineStructure = enum.auto() + PlaneStructure = enum.auto() BoxStructure = enum.auto() SphereStructure = enum.auto() CylinderStructure = enum.auto() + PolySlabStructure = enum.auto() # Bounds BoundConds = enum.auto() @@ -91,7 +93,6 @@ class NodeType(blender_type_enum.BlenderTypeEnum): PECBoundCond = enum.auto() PMCBoundCond = enum.auto() BlochBoundCond = enum.auto() - PeriodicBoundCond = enum.auto() AbsorbingBoundCond = enum.auto() # Monitors diff --git a/src/blender_maxwell/node_trees/maxwell_sim_nodes/nodes/inputs/unit_system.py b/src/blender_maxwell/node_trees/maxwell_sim_nodes/nodes/inputs/unit_system.py index a290f4d..58975f9 100644 --- a/src/blender_maxwell/node_trees/maxwell_sim_nodes/nodes/inputs/unit_system.py +++ b/src/blender_maxwell/node_trees/maxwell_sim_nodes/nodes/inputs/unit_system.py @@ -3,7 +3,7 @@ from ... import sockets from .. import base, events -class PhysicalUnitSystemNode(base.MaxwellSimNode): +class UnitSystemConstantNode(base.MaxwellSimNode): node_type = ct.NodeType.UnitSystem bl_label = 'Unit System' @@ -31,6 +31,6 @@ class PhysicalUnitSystemNode(base.MaxwellSimNode): # - Blender Registration #################### BL_REGISTER = [ - PhysicalUnitSystemNode, + UnitSystemConstantNode, ] BL_NODES = {ct.NodeType.UnitSystem: (ct.NodeCategory.MAXWELLSIM_INPUTS)} diff --git a/src/blender_maxwell/node_trees/maxwell_sim_nodes/sockets/base.py b/src/blender_maxwell/node_trees/maxwell_sim_nodes/sockets/base.py index 9baaa3e..fa7fc3e 100644 --- a/src/blender_maxwell/node_trees/maxwell_sim_nodes/sockets/base.py +++ b/src/blender_maxwell/node_trees/maxwell_sim_nodes/sockets/base.py @@ -286,8 +286,7 @@ class MaxwellSimSocket(bpy.types.NodeSocket): """ self.display_shape = ( 'SQUARE' - if self.active_kind - in {ct.FlowKind.LazyValueFunc, ct.FlowKind.LazyValueRange} + if self.active_kind in {ct.FlowKind.Array, ct.FlowKind.LazyValueRange} else 'CIRCLE' ) + ('_DOT' if self.use_units else '') @@ -882,7 +881,6 @@ class MaxwellSimSocket(bpy.types.NodeSocket): { ct.FlowKind.Value: self.draw_value, ct.FlowKind.Array: self.draw_array, - ct.FlowKind.LazyValueFunc: self.draw_lazy_value_func, ct.FlowKind.LazyArrayRange: self.draw_lazy_array_range, }[self.active_kind](col) @@ -1000,16 +998,6 @@ class MaxwellSimSocket(bpy.types.NodeSocket): col: Target for defining UI elements. """ - def draw_lazy_value_func(self, col: bpy.types.UILayout) -> None: - """Draws the socket lazy value on its own line. - - Notes: - Should be overriden by individual socket classes, if they have an editable `FlowKind.LazyValueFunc`. - - Parameters: - col: Target for defining UI elements. - """ - def draw_lazy_array_range(self, col: bpy.types.UILayout) -> None: """Draws the socket lazy array range on its own line. diff --git a/src/blender_maxwell/node_trees/maxwell_sim_nodes/sockets/basic/expr.py b/src/blender_maxwell/node_trees/maxwell_sim_nodes/sockets/basic/expr.py index 5f3cd8c..e28b726 100644 --- a/src/blender_maxwell/node_trees/maxwell_sim_nodes/sockets/basic/expr.py +++ b/src/blender_maxwell/node_trees/maxwell_sim_nodes/sockets/basic/expr.py @@ -69,15 +69,13 @@ class ExprBLSocket(base.MaxwellSimSocket): #################### @property def value(self) -> sp.Expr: - expr = sp.sympify( + return sp.sympify( self.raw_value, locals={sym.name: sym for sym in self.symbols}, strict=False, convert_xor=True, ).subs(spux.ALL_UNIT_SYMBOLS) - return expr - @value.setter def value(self, value: str) -> None: self.raw_value = sp.sstr(value)