How can one research system search every architecture without imposing one architecture’s assumptions on the rest?
Architecture-neutral does not mean architecture-blind
A universal reduction mandate should not favour one architecture family. It must still read the structure, state, movement and execution of the system in front of it before deciding which candidate paths are physically meaningful.
Keep the objective universal and make the search grammar conditional on observed architecture-specific truth.
Architectures carry intelligence in different physical forms
Dense transformers, sparse expert systems, state-space models, recurrent systems and multimodal hybrids can differ in where parameters reside, which state becomes workload-dependent and how information moves through execution.
The same nominal intervention can therefore have different complete-system effects. Removing inactive computation is not identical to removing resident state. Compressing persistent weights does not automatically reduce a dominant runtime cache. A valid search must resolve those distinctions.
The objective can remain universal
The common question is physical: what is the smallest state this specific intelligence can occupy without crossing its defined quality boundary? That mandate does not require a universal shortcut.
Instead, it provides a stable comparison frame. Every architecture is measured against its own baseline, workload and evidence. The result is not which architecture is universally best, but how far each target can validly move from the physical form it currently occupies.
Universal mandate. Architecture-specific truth. No universal shortcut.
Use an adaptive search grammar
The research system should first identify the components, dependencies, state transitions, execution paths and physical allocations that exist. Candidate families can then be generated from that map rather than from a fixed catalogue.
As evidence exposes new interactions, the search grammar should expand. A previously independent reduction may need composition testing; an execution change may reveal a different memory bottleneck; a blocker may redirect the search toward another layer.
Judge the composition, not the parts
Individually valid changes can interfere when combined. Their quality effects may overlap, their memory savings may double-count the same allocation and one transformation may change the conditions assumed by another.
Architecture-aware composition is where candidate reductions become a system state. The composed candidate must be remeasured and revalidated as one physical whole.
Leave room for what comes next
A research system tied to today’s dominant architecture risks becoming obsolete with it. The public taxonomy should therefore remain open to new forms of recurrence, sparsity, memory, modality and hardware–software composition.
Architecture neutrality is ultimately a refusal to confuse the current map with the territory. The objective persists even as the structures carrying intelligence change.
References
Primary context.
Publicly traceable.
Attention Is All You Need
Vaswani et al.Primary paper introducing the Transformer architecture.
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Gu and DaoPrimary paper illustrating a selective state-space architecture with different execution and state properties.
Switch Transformers
Fedus, Zoph and ShazeerPrimary paper illustrating sparse expert routing and the distinction between total and active parameters.
Cite this note
VAMANIR (25 July 2026). “Architecture-neutral does not mean architecture-blind.” Research Note 005, v1.0. https://www.vamanir.com/research/notes/architecture-neutral-not-blind