Mark's Code Fractal   

Prev Home Next

Mark's Code Fractal Development Log

At this point, the focus is on getting the core of the Fractal engine up and running on the latest and greatest code base, but in multi-threaded, re-entrant, distributed form suitable for deployment within a large multi-core web server environment. The code base was both JDK17 and JDK25 compatible, but it is now being built using Eclipse Temurin JDK 25.

The security model has been refactored and is ready to have the enforcement code written. The 2.0.13 implementation of code customization through the SchemaTweak and TableTweak sub-objects of the schemas and tables in a model, and the parsing and compiling of the schematweak and tabletweak GEL verbs has been added to the GEL compiler. I expected the tweak task to take longer than it did.

2026-04-13 00:50 CST
Added the verb DefSchemaTables to 2.0.13 which iterates the tables defined by that particular schema. The 3.1 I[CFSec|CFInt|CFBam]Schemas have had getTableInfo() and getCompositeTableInfo() added to them, with the latter used for the top-most schema to pass down a consolidated list of table information to the CFSec implementation of schema initialization.
2026-04-12 17:50 CST
The security and table bootstraps finally clean compile, but I'm not satisfied yet - I want the entire set of table security information to commit as a single transaction on startup.
2026-04-11 22:00 CST
In CFBam model, SchemaDefObj objects now can have a SchemaTweak set of named objects underneath, and a TableObj can have TableTweak named objects underneath. The custom SAX parser for the models has been updated accordingly. Note that the tweaks are not pulled into a SchemaDef when importing a SchemaRef. This allows each SchemaDef to redefine the SchemaTweak and TableTweak instances by having "name collisions" with the referenced schemas. Then when processing the schematweak and tabletweak macros, the definition hierarchy is probed accordingly. In the knowledge base, "schematweak <TweakName>" macro invocations while on a SchemaDef object will find the appropriate set of tweaks in the SchemaRefs for the SchemaDef, trying to build the specializations from the lowest-level model on up. "tabletweak <TweakName>" macro invocations while on a Table object will find the appropriate set of tweaks in the DefSchema hierarchy for the table, eventually stopping at the table with a null DefSchema. As the tweaks are found, they are added to the beginning of the processing list of tweaks. When processing an appropriately created ArrayList of tweaks, the encounter of a tweak which specifies ReplacesInherited="true" will cause the GEL compile buffer to reset to empty. Otherwise, each tweak's GEL text is just appended to the GEL compile buffer as the tweaks are iterated through. Eventually, if the GEL compile buffer is not empty, the GEL is compiled in a try-catch block reporting any compile errors accordingly and expanded to produce the return result. If the GEL compile buffer is empty, an empty string is the return result.
2026-04-11 16:30 CST
msscode 2.0.13 exercises the SchemaTweak and TableTweak entries added to the 3.1 CFSec model, navigating the object hierarchy successfully to produce the output. MCF 3.1 clean compiles.
2026-04-11 11:15 CST
The builtin expansion verbs schematweak and tabletweak have been excercised for the null/no-such-definition case by modifying the knowledge base to invoke such expansions.
2026-04-11 11:10 CST
The builtin expansion verbs schematweak and tabletweak have been added to the 2.0.13 GEL compiler, and expect the balance of the macro to be the tweak name to be expanded. Searching of DefSchema and SchemaRef inheritence is done through code, and the tweak objects should not be copied when importing a schema reference.
2026-04-11 01:50 CST
The MCF v3_1 CFBam.Tweak objects have the same changes as v2_13 propagated forward to them. There is now a unique index key including the Def columns.
2026-04-11 01:45 CST
The v2_13 CFBam.Tweak objects now have an index key that includes the Def columns for use in merging the definitions on load in the runtime code that is soon to be written.
2026-04-11 01:40 CST
The v2_12 models were updated to tweak the v2_13 CFBam.Tweak object to have an index key that includes the Def columns for use in merging the definitions on load.
2026-04-07 09:40 CST
Almost forgot to ensure there are Cluster-level security entries for the Tenant-level security entries so that you can have a cluster-only superuser for the cluster's tenants instead of having to use the system level administration account for maintenance.
2026-04-05 02:30 CST
The initial coding of the security bootstrap groups, members, and subgroup inclusions is complete, now that a public group has been added.
2026-04-05 01:20 CST
The JPA security priming code has been written, but not tested. I think I should test it before I clone a big chunk of it to the buff layer.
2026-04-04-22:20 CST
The security framework priming is proceeding apace; I believe the code for the Sys level objects is now complete, and ready for the Tenant and Cluster scoped variants
2026-04-04 12:30 CST
With great pleasure I report that not only does the entire set of v3_1 fractal code now clean compile again, but the bug in the cfsecramtest that I knew about has been fixed - there is now only one user record for the admin user in the data set during the test instead of the erroneous two that I used to see.
2026-04-04 02:45 CST
Clean compile of v3_1 up to cfsecram, which needs updating to deal with the new models and relationships
2026-04-02 10:10 CST
Wrapped the resolution of named sub-objects in try-catch of Throwables as the fact you cannot convert a random string to a specific type of data is not an error in this case, but a common occurrence meaning that that particular lookup cannot be resolved using the string provided
2026-04-01 20:40 CST
You can now use any kind of column you like as the final column of a LookupIndex for a Table, and the strings will be auto-converted to the appropriate intrinsic type for probing the lookup index. I think I might need to wrap and ignore conversion exceptions, though, because a bad conversion just means the string can't possibly resolve to that particular table reference
2026-03-30 07:15 CST
Have been working on adding some new objects to the SecUser group of objects, but doing so exposed issues with the knowledge base' implementation and handling of some edge cases. Many of those edge cases have been corrected, but it isn't a clean-compiling check-in yet.
2026-03-27 01:10 CST
Version 3.1 of Mark's Code Fractal has added CFBam.Tweak and subclasses CFBam.TableTweak and CFBam.SchemaTweak, brought forward from the 2.0.13 models. I don't expect to be bringing forward and modernizing the related custom code for some time yet. Quite some time.
2026-03-26 23:50 CST
Version 2.0.13 of MSS Code Factory has added CFBam.Tweak and subclasses CFBam.TableTweak and CFBam.SchemaTweak as components of the obvious objects. These collect the customization tag information from parsing the model. Next up is modifying the 2.0.13 CFBam custom parser to accept the new data, and wiring the MssCF custom parser and actions to process the new data. The verbs 'tweak', 'tweakschema', and 'tweaktable' will be added to the custom Gel compilation, similar to the way the hardcoded customizations are handled now, but dynamically. Then a two-phase approach is used, where the rules are loaded priming the tweak name sets so they can be validated during the model parsing phase. Then and only then can I begin using this new feature that was originally slated for 3.1. I just decided it would be easier to implement for 2.0.13 than it would be to create all the custom expansion verbs and columns for 2.0.13 that I'd need for this new ruleset. I never did like the way I was handling customization... too tightly bound to the knowledge base being processed. Once I've verified the code is falling together reasonably well, I'll migrate the new objects and their relationships to 3.1 to carry the feature change forward.
2026-03-26 17:10 CST
Ready to redo the security initialization code, but this time fully fleshed with the new model in place
2026-03-26 03:30 CST
The CFSec model changes are tentatively complete. The code used to prime the database now has to be refactored and reworked to support the new model, after which I can start working on the code that enforces the security constraints.
2026-03-26 00:20 CST
Cleaned up the RAM errors because I didn't want to go to bed on compile errors. So even though the CFSec changes aren't complete yet, the code all clean compiles at least.
2026-03-25 20:55 CST
Finally got most of the CFSec packages building with the partial remodelling of the security tables in place; there were issues with relationships not being fully orthogonal in the model, preventing them from being processed as Superior relationship code. Now there are issues with the RAM storage layer, but I'll deal with that some other day.
2026-03-25 10:05 CST
Snapshot of CFSec (a work in progress) with tweaks to the CFBam.SecScopeEnum to support None, System, Global, Cluster, Tenant, SystemGroup, ClusterGroup, and TenantGroup. There is no GlobalGroup, as it is a SystemGroup with no read access enforcement restriction.
2026-03-25 04:45 CST
I've realized the real problem I'm having with the code produced is that the Superior relationship classification has to accept any unique index as the ToIndex of the relationship, not just primary indexes. This requires a series of 2.13 engine code changes, and possible knowledge base changes for the buff and jpa modules.
2026-03-20 02:00 CST
Added security model documentation for 3.1, sketchy and basic though it may be
2026-03-15 18:40 CST
Make Mark Stephen Sobkow's exception a specific clause in the gpl-3.0* and lgpl-3.0* license files for the various packages and projects. Clean up licensing, integrate resource resolution for 2.12, 2.13, and 3.1 via mcf.markhome.server URLs, and generally clean up the 2.13 and 3.1 code base.
2026-03-15 13:30 CST
The licensing has been coordinated and synchronized between the 2.13 and 3.1 releases, both of which use Apache V2 licensing for the cfsec and cfint jars and packages, GPLv3 with classpath, static linking, and my own special exceptions for cfbam and cfcore. The various cflib packages use LGPLv3 with classpath, static linking, and my own special exceptions. I specifically allow you to leverage newer features of Java like the ability to distill a bundle of jars down to a single executable jar.
2026-03-09 15:25 CST
Finished migrating to the nexus3.mcf.markhome.server-hosted repository for publication of code builds, brought the commons-codec references in 2.13 up to date.
2026-03-08 23:05 CST
Cleaned up more of the details of the migration back to mcf.markhome.server/server.markhome.mcf from msobkow.github.io/io.github.msobkow. At this point I think I've got all the kinks and details worked out for the 3.1.42 builds I've been working on, using github.io for the documentation publication, .xsd file distribution, and remote git repository provisioning for offsite backup of my work (hey, when you spend 25-30+ years banging away obsessively on the same idea, reworking pieces and redoing things until you're actually proud of them and the cleanliness of the code, you don't want to be without at least one viable backup at all times! I even carry a USB stick of the latest backups when I leave the building for an extended time, just in case there's a fire or something and I lose my machine completely.) The local services provide a nexus3 project-hosted repository and a project-public group that includes the project-hosted and maven-central repos, with all the pom.xml's set up to publish to the project-hosted repository. Obviously you're expected to configure your .m2/settings.xml to specify that the project-public group repo should be used for resolving maven artifacts. Add additional repos (hosted or groups) to the project-public repository as needed. The documentation now looks identical regardless of whether it is resolved as https://msobkow.github.io/, http://mcf.markhome.server/, or via direct opening via right-mouse-click on the index.html in the document root and opening it with a browser. Roughly 4GB of source code was redirected today, so I think that's quite enough for a day's work.
2026-03-08
Shifting back to server.markhome.mcf from io.github.msobkow for the project due to issues with publishing the maven builds - it tries to publish them to io.github.
2026-03-03 21:55
Set up a Sonatype nexus3 service and enhanced the rule base to leverage the new "SchemaHostName" verb to set the target server to nexus3.$lower SchemaHostName$ with a distribution repository target of $lower SchemaHostName$-hosted for the publication of the packages. Then you set up a $lower SchemaHostName$-public group that includes the -hosted first, and then the MavenCentral cache as a fallback for non-local package resolution. That way the local repository can resolve the io.github.msobkow packages before any msobkow.github.io addresses can be probed; a simple host name alias for my system in /etc/hosts takes care of intercepting nexus3.msobkow.github.io in my case. I've also resurrected the use of build number arguments, so you'll want to coordinate the manual custom pom.xml specifications with the build number specified for producing the fractal code.
2026-02-28 14:15
Refactored schema table security, added restore[tablename] and mutate[tablename] for tables with history and mutable tables, respectively.
2026-02-28 09:05
Fleshed out the table bootstrap security for the top-level Cluster based security that decides if you ever have permission to do something. For tables that have Tenant security, I next need to populate the default security sysadmin permissions for the system tenant. Log is at 2026-02-28-09h05-BamTestsWithSecurityBootstrappingRunlog.txt
2026-02-27 05:00
Added bin-v3_1/InstallRepos31.bash and bin-v2_13/InstallRepos213.bash to create the default install locations if install locations haven't been set and configured, and to install all of the repositories for the release as appropriate
2026-02-27 01:00
Update of CFSec with initial bootstrap of CFInt and CFBam databases, runlog is here: 2026-02-27-01h00-UpdateCFSecInitCFIntCFBamRunlog.txt
2026-02-27 00:30
Finally had an initial bootstrap run of the bootstrap Cluster, "system" Tenant in the Cluster, admin SecUser, the SysCluster entry that makes the bootstrap data the system singleton, and the SecSession record that tracks the management of the whole operation with begin and end LocalDateTime values on insert and update-to-close the session. The initial runlog of the core CFSec bootstrap process is here: 2026-02-27-00h25-InitialBootstrapRunlog.txt
2026-02-26
Resolved the issue with dependency injections to my satisfaction, and in the process of debugging the security bootstrap code and JPA source. 3,977,931 lines of code in the 3.1 code base between hand-written and fractal code, with the vast majority being fractal.
2026-02-21 AM
Get rid of AspectJ and spring-aspects next; dynamic resolution via ApplicationContext is relatively trivial, and much easier to work with.
2026-02-21 Early AM
Keep AspectJ because it should work after all, but I don't actually need to use AspectJ directly, and shouldn't be referencing it directly in my POMs, apparently. Instead, I need to rely on Spring Boot annotations that exist by default but which get runtime AspectJ style initialization on construction via new() operations after the instance is fully created, but before any actual runtime code is used to invoke the object following the new() operation that created it. So by creating a POJO in a separate schemajpahooks package whose classes are intended to be POJOs merely annotated by @Configurable instead of being full @Components. We don't want Spring to create instances and wire a web of them, we want POJO code to specify configuration layers before JPA is initialized and have their @Autowired references resolved on dynamic construction, not on boot layout processing. Currently there is one massive hooks object for the Schema that references all the Spring objects that are expected to exists for a JPA framework implementation. This ensures that the entire set of objects is guaranteed to have their singletons instantiated when the schema first constructs the originally-null [Schema]JpaHooksSchema member during the first getSchemaHooks() invocation after it's instantiation. I think what I have to do next is shift the pom.xml references to the spring-aspects dependency to the final main that initializes everything, because the maven processing consolidates all the class loaders into a single .jar and loader as far as I'm aware; I could be wrong. If I am wrong about that, just specifying spring-aspects was supposed to get the autowiring working, but it doesn't. Obviously I'm missing something...
2026-02-20 AM
AspectJ isn't going to do what I want! I need to do a post-construction initialization of the @Autowired references in my JPA Schema wiring/hook POJOs because those objects get constructed _before_ Spring initialization takes place. The initialization of those @Autowired references has to happen _after_ Spring is initialized and has resolved all the "normal" Spring objects in the JPA layer. By default, AspectJ does it's initialization by wiring itself as a "wedge" constructor with the ajc compiler such that the wedge constructor gets invoked instead of the default constructor, invokes the default constructor internally, and then does the autowiring. That's not what I need at all. I need to figure out how to do _dynamic_ bean resolution by name and interface type. I know it can be done 'cause it's been done on prior projects; I just need to relearn how Spring does it (besides, if I don't even remember what I'm looking for, it's probably an API that's changed by now, so I'm better off relearning 4.0 information.) Obviously I then need to remove @Autowired from the POJOs and the spring-aspects annotations, refactor those formerly @Autowired members as AtomicReference<Interface|Class>, and wrap their use in getter methods that use dynamic spring resolution mechanisms to locate the bean by name and by type and wire it to the member using ar.get() and ar.set(expected,value) in the getter method to ensure only one setting of the resolution gets kept by this attribute. Easy-peasy. Probably have it done today, now that a plan is in place.
2026-02-19
CANCELLED It turns out that JPA requires binding the POJOs that use @Autowire directives using AspectJ under the hood. The key is to add the spring-aspects package to your pom to anything making such references. This allows AspectJ to handle the autowiring for beans that have been annotated with @Configurable (you should specify whether to configure by name or by class; in my case virtually all my bindings are by name, but I haven't added that specification yet - I just read about it a few minutes ago. Once you've done that, you specify a bean in your main executable projects that rely on such annotate POJOs to initialize and wire the AspectJ components. Finally, at runtime, your java command has to specify that spring-aspects.jar is to be loaded as well as the *-spring-boot.jar that was produced as the unified application jar; the easiest way to ensure that is done is to wrap all your *-spring-boot.jars with bash scripts that pass along the command line arguments to the java command, but ensure that the command line arguments for launching the application are consistent. Even applications which don't rely on JPA should do so; this is a low-priority todo and has been for some time, but this recent change has made it important to take care of as the final step of the AspectJ wiring.
2026-02-13:14
ON HOLD - Requires new approach based on 2026-02-20 AM notes - Bootstrap code for priming the CFSec database with the "system" cluster, "system" tenant, "sysadmin" user with initial password "ChangeOnInstall", the SysCluster record, and the SecSession for the bootstrapped security data has been added to the Buff and JPA implementations, with a "bootstrapSchema()" method added to the schema interfaces. Atomic references for the SysClusterId, SysTenantId, and SysAdminId have been added and are initialized by the security portion of the bootstrapSchema() implementations for CFSecBuffSchema and CFSecJpaSchema. This is very much a work in progress - now that I've got the atomic references and their accessors, I can code initial security constraints for the table accessor methods that give the admin user global access, and pass in a security object during the bootstrap process that is based on those values. I want to get the initial security framework wired back in at this point in time as well, so that I've dealt with the special cases needed for the bootstrap processing. The SysCluster record and the actual system Cluster have to be readable by anybody, but trying to read any other Cluster should be secured. The existence of valid user ids has to be verifiable, but the SecUser table itself shouldn't be globally readable. SecSession should be tightly restricted, such that only the SysAdmin user can access it other than in-security-process code. The exceptions that were being thrown by CFSecRamTest have been cleaned up, but there are still two CFSecSecUser entries where there should only be one.
2026-02-10
It turned out that no syntax errors cropped when rebuilding with JDK17, even though I thought the use of final statics in interfaces was a JDK25 feature. The downgrade from JDK25 syntax to JDK17 syntax is with the intent of using GWT for Spring for the user interfaces, so I may not be able to continue with the static finals when I'm relying on the GWT Java processor that translates pure JDK17 code to JavaScript in the client. I have some ideas on how to go about mapping my JavaFX code to GWT code... and getting dynamic layout flow along the way! It's just fortunate GWT at least supports JDK17; when I was first exposed to it's commercial relative, only JDK8 code was supported.
Older
A simple test program similar to cfxxxjpatest called cfxxxramtest makes sure the code all wires together successfully and that the empty tables can be find-all'ed.
Older still
The base for the DbKeyHash implementations now uses a 64-bit ClusterCode instead of a 32-bit Machine Id. This change is more than name, but in fundamental meaning, because in the Code Fractal world, a ClusterCode is a randomly generated value stored in the singleton SysCluster table. This value is randomly generated on system install, and in the future will be submitted to a central server for registration. But the only way I could automate such processing right now is through the use of my personal email account with one of my ISPs, and I don't think they'd appreciate the potential traffic flood (i.e. Part of setup is sending the registration information encrypted by some known public key so that my email account can decrypt it, verify the ClusterCode is unique and accept the registration, or detect the ClusterCode collision and find a free random one to use instead, and conditionally accept the registration with the proviso that the node change it's ClusterCode to the assigned id. That way 99% of the time the _client_ produces the ClusterCode values, and the server only has to _register_ them. The internal keys used for the "system" Cluster and "system" tenant are irrelevant, and will be unique to each cluster, as will the id of the "admin" account for the cluster. No easy "defaults" other than the initial "admin" password being a known string that gets changed on initial login.

In the near future after the bootstrap code is in both the cfxxxjpatest and cfxxxramtest code and has been used to successfully bootstrap data for those implementations, the following items are on the "todo" list for the relatively near future. Don't worry, there is plenty more coming after that. I don't expect to be done coding my vision until the end of 2026Q3 at earliest, probably well into 2027Q1.