Minimal 3D Game Asset Encoding Techniques

A comprehensive catalog of techniques for achieving the smallest possible 3D video game assets, inspired by demoscene practices. Each technique includes compatibility notes for building effective combinations.

Table of Contents

1. Data Dynamism Spectrum (Immediate ↔ Retained)
2. Interactive Shader Development & Debugging
3. S-Expression Based Shader Language
4. Engine Compatibility Notes
5. Observed Patterns & Principles
6. Technique × Spectrum Compatibility Matrix
7. Hardware & Software Requirements
8. Offline / Online & Interactivity
9. Reference Appendix
10. Geometry Representation
11. Procedural Generation
12. Texture Techniques
13. Rendering Paradigms
14. Compression & Encoding
15. LOD & Streaming
16. Shader-Based Techniques
17. Animation Techniques
18. Lighting & Materials
19. Audio (Bonus)
20. Code Size Reduction

Data Dynamism Spectrum (Immediate ↔ Retained)

This section describes the fundamental axis of how static your data is - from fully retained (load once, GPU owns it forever) to fully immediate (CPU rebuilds everything every frame). This spectrum affects which techniques are compatible, where bottlenecks appear, and how debuggable your system is.

The Spectrum

RETAINED                                                          IMMEDIATE
(Static)                                                          (Dynamic)
    |                                                                  |
    |  D1  |  D2  |  D3  |  D4  |  D5  |  D6  |  D7  |  D8  |  D9  |  D10 |
    |      |      |      |      |      |      |      |      |      |      |
   GPU    GPU    GPU    GPU   Mixed  Mixed   CPU    CPU    CPU    CPU
  owns   owns   owns   drives        leans  drives builds  builds software
  world  scene  meshes render        CPU    GPU    GPU     all    renders
  graph  graph  only   cmds                 cmds   data    data   all

D1: Fully Retained World Scene Graph on GPU

Description: Entire world representation lives in GPU memory. GPU-driven rendering with minimal CPU involvement. Scene graph, transforms, visibility - all GPU-side.

Characteristics:

• CPU uploads world once (or streams chunks)
• GPU handles culling, LOD selection, draw call generation
• Compute shaders manage scene updates
• CPU mainly handles input and high-level game logic

Examples:

• Nanite (UE5)
• GPU-driven rendering pipelines
• Modern AAA engines

Works well with:

• G6 (Virtualized geometry)
• L1 (GPU-selected LOD)
• Massive static worlds
• Persistent multiplayer environments

Does not work well with:

• Rapid prototyping
• Debugging (opaque GPU state)
• Highly dynamic worlds
• Simple projects (massive infrastructure)

Compression relevance:

• Heavy compression worthwhile (amortized over long retention)
• Streaming compression (Basis, Draco) makes sense
• GPU-decodable formats preferred

Debugging characteristics:

• Very difficult - state lives on GPU
• Need GPU debuggers (RenderDoc, PIX)
• Performance issues can be hidden/amortized
• Hard to reason about what's actually happening

D2: Retained Scene Graph, CPU Transform Updates

Description: Scene graph and meshes retained on GPU, but CPU updates transforms each frame and manages object lifetime.

Characteristics:

• Meshes uploaded once, retained
• CPU maintains parallel scene representation
• Transform matrices sent per-frame
• CPU does visibility culling, sends draw calls

Examples:

• Traditional game engines (Unity, older UE)
• Most shipped games historically

Works well with:

• G3 (Standard meshes)
• Standard game logic
• Reasonable dynamism
• Established workflows

Does not work well with:

• Extreme object counts (draw call limits)
• Per-vertex animation
• Procedural geometry

Compression relevance:

• Mesh compression still valuable
• Texture compression standard
• Less pressure than D1 (more flexibility)

Debugging characteristics:

• Moderate - can inspect CPU-side scene
• Draw calls visible and countable
• Transform issues debuggable
• Still need GPU tools for rendering bugs

D3: Retained Meshes, Dynamic Materials/Textures

Description: Geometry stable, but materials, textures, or shader parameters change frequently.

Characteristics:

• Mesh data static
• Frequent uniform/constant buffer updates
• Texture swapping or streaming
• Shader parameter animation

Examples:

• Games with dynamic weather/time of day
• Material parameter animation
• Texture streaming systems

Works well with:

• T4 (Virtual texturing)
• Dynamic lighting
• Visual variety without geometry cost

Does not work well with:

• Static baked lighting (conflicts with dynamic materials)
• Extreme material counts

Compression relevance:

• Texture compression critical (streaming bandwidth)
• Mesh compression still applies

Debugging characteristics:

• Moderate - geometry stable baseline
• Material bugs more visible
• Can diff against known-good state

D4: GPU-Driven with CPU Feedback

Description: GPU does heavy lifting but CPU reads back results for game logic (occlusion queries, physics readback).

Characteristics:

• GPU generates visibility data
• CPU reads back for AI, physics, gameplay
• Latency-sensitive synchronization
• Complex pipeline

Examples:

• GPU occlusion culling with CPU readback
• GPU physics with gameplay integration
• Async compute patterns

Works well with:

• Large worlds needing CPU-side knowledge
• AI that needs visibility info
• Complex gameplay systems

Does not work well with:

• Low latency requirements (readback delay)
• Simple games (complexity overhead)
• Debugging (async timing issues)

Compression relevance:

• Readback bandwidth matters
• Compressed intermediate formats possible

Debugging characteristics:

• Difficult - async timing issues
• GPU/CPU sync bugs subtle
• Need careful frame analysis

D5: Mixed Mode - Instanced Static + Dynamic Objects

Description: Static world retained, dynamic objects handled more immediately. Common practical compromise.

Characteristics:

• Background/environment retained
• Characters, items rebuilt/updated frequently
• Clear separation of concerns
• Hybrid data flow

Examples:

• Most modern games
• Open worlds with dynamic entities
• Practical production approach

Works well with:

• Most standard techniques
• Reasonable performance
• Flexible content

Does not work well with:

• Fully dynamic worlds
• Extreme consistency requirements

Compression relevance:

• Static content: heavy compression
• Dynamic content: lighter/faster compression

Debugging characteristics:

• Moderate - can focus on one half at a time
• Clear system boundaries help

D6: CPU-Driven Rendering, Retained Buffers

Description: CPU decides everything each frame but reuses uploaded buffer data. Traditional immediate-mode thinking with some retention.

Characteristics:

• CPU builds command lists each frame
• Buffers persist but CPU controls usage
• Full CPU visibility into what's rendered
• Explicit resource management

Examples:

• Explicit APIs used "simply" (Vulkan/D3D12 without GPU-driven)
• Older immediate-mode thinking with modern API
• Teaching/learning graphics

Works well with:

• Debugging (CPU sees everything)
• Explicit control
• Understanding what's happening

Does not work well with:

• Extreme draw counts
• Massive worlds
• Full GPU utilization

Compression relevance:

• Upload cost visible per-frame
• Compression must be fast to decode

Debugging characteristics:

• Good - CPU state fully inspectable
• Can trace every decision
• Performance costs visible and attributable

D7: Streaming Geometry (Partial Retention)

Description: Continuously stream geometry based on camera/player position. Hybrid retention with constant churn.

Characteristics:

• Working set retained
• Background streaming in/out
• Memory budget management
• Position-dependent loading

Examples:

• Open world streaming
• Flight simulators
• Google Earth style

Works well with:

• L5 (Streaming)
• Massive worlds
• Limited memory

Does not work well with:

• Random access (teleportation)
• Fast movement (pop-in)
• Small contained levels

Compression relevance:

• Streaming bandwidth critical
• Progressive formats valuable
• Decompression speed matters

Debugging characteristics:

• Moderate - streaming adds complexity
• Loading bugs hard to reproduce
• Position-dependent behavior

D8: Per-Frame Geometry Generation (GPU)

Description: GPU generates geometry each frame via compute shaders. Data flow: parameters → compute → render.

Characteristics:

• Minimal CPU upload (just parameters)
• Compute shaders build vertex/index buffers
• Marching cubes, procedural mesh, etc.
• GPU-only intermediate data

Examples:

• Voxel terrain (GPU marching cubes)
• Procedural geometry
• Particle mesh generation

Works well with:

• P6 (Compute shader generation)
• G4 (Voxels)
• Highly dynamic geometry
• Destructible environments

Does not work well with:

• CPU physics on generated geometry
• Traditional content pipelines
• Debugging (GPU-only state)

Compression relevance:

• Parameters compress well (tiny)
• No mesh compression needed
• Algorithm IS the compression

Debugging characteristics:

• Difficult - geometry exists only on GPU
• Need compute shader debugging
• Can't easily inspect intermediate state

D9: Per-Frame Full Rebuild (CPU → GPU)

Description: CPU builds all render data every frame, uploads to GPU. True immediate mode.

Characteristics:

• No retained geometry
• Full CPU control and visibility
• Upload bandwidth is the bottleneck
• Every frame starts fresh

Examples:

• Dear ImGui (2D)
• Debug visualization systems
• Prototyping
• Some CAD/visualization tools

Works well with:

• Debugging (perfect visibility)
• Rapid iteration
• Highly dynamic content
• Simple mental model

Does not work well with:

• Large data volumes
• Performance-critical rendering
• Complex meshes
• Production games

Compression relevance:

• Compression often counterproductive (decode cost)
• Bandwidth to GPU is the limit
• Simple formats preferred

Debugging characteristics:

• Excellent - full CPU visibility
• Every frame reproducible
• No hidden state
• Performance issues immediately obvious

D10: CPU Software Rendering

Description: CPU does all rendering, writes directly to framebuffer. Ultimate immediate mode.

Characteristics:

• No GPU involvement
• Full programmatic control
• Single address space
• Every pixel computed explicitly

Examples:

• Demoscene software renderers
• Embedded systems
• Educational projects
• Fallback renderers

Works well with:

• Learning
• Extreme control
• No driver bugs
• Deterministic rendering

Does not work well with:

• Performance
• Modern resolutions
• Complex effects
• Shipping games (usually)

Compression relevance:

• Compression must be CPU-decodable
• Memory bandwidth different constraints
• May want uncompressed for speed

Debugging characteristics:

• Perfect - everything in CPU memory
• Can inspect any pixel's computation
• Single-step through rendering
• Print statements work

Why Immediate Mode Matters for Development

You noted preference for immediate mode for several reasons that are worth expanding:

1. Performance Transparency In retained mode, costs are amortized and hidden:

• Uploading a 1M triangle mesh takes 50ms... but it happened 30 seconds ago
• That spike in your profiler? Garbage collection of a released buffer from 3 frames ago
• Draw call batching makes individual object costs invisible

In immediate mode:

• Frame N's costs are incurred in Frame N
• Profiler shows actual work being done NOW
• No "well it was fast because it was cached" surprises

2. State Machine Clarity

// Retained mode question: "What is the GPU rendering right now?"
// Answer: Whatever combination of:
//   - Scene graph state (when was it last modified?)
//   - Pending uploads (are they done?)
//   - Cached command buffers (from when?)
//   - Material overrides (which are active?)

// Immediate mode answer: "Whatever I told it to render this frame."

3. Debugging Workflow

// Immediate mode debugging:
if (frame == 1234) {
    printf("About to draw object X at position %f,%f,%f\n", ...);
    // Set breakpoint here, inspect everything
}

// Retained mode debugging:
// "The object was uploaded at frame 100, 
//  its transform was last set at frame 1200,
//  the GPU is using command buffer from frame 1232,
//  the actual draw happens... somewhere in the driver"

4. Effect Implementation When implementing a new effect in immediate mode:

• Write code that computes the effect
• See result
• Done

When implementing in retained mode:

• Figure out where in the scene graph to hook
• Determine what data needs caching
• Handle invalidation when dependencies change
• Manage lifetime of cached data
• Actually write the effect
• Debug why cached data is stale

5. Reproducibility Immediate mode: Same inputs → Same outputs (per frame) Retained mode: Same inputs → Different outputs (depending on history)

Immediate Mode Techniques Expanded

Since you want to expand on immediate mode, here are techniques that work well in that paradigm:

IM1: Ray Marching SDFs (Fully Immediate)

Each pixel independently computed from scratch. No retained geometry. Perfect immediate mode.

Implementation pattern:

// Every frame, every pixel:
for each pixel:
    ray = computeRay(pixel, camera)
    for steps in range(MAX_STEPS):
        p = ray.origin + ray.dir * t
        d = sceneSDF(p)  // Evaluate ENTIRE scene
        if d < EPSILON: shade and return
        t += d

Why it's immediate-friendly:

• Scene defined by function, not data
• No uploads, no buffers, no state
• Change scene function → immediate visual change
• Add object = add term to SDF function

IM2: Compute-to-Render Direct Pipeline

Generate geometry in compute, render same frame, discard.

Pattern:

Frame N:
  1. Dispatch compute (params → vertices)
  2. Barrier
  3. Draw generated vertices
  4. Discard vertex buffer (or ring-buffer reuse)

No retention, full regeneration each frame.

IM3: Uniform-Driven Procedural Geometry

Vertex shader generates positions from uniforms + vertex ID.

// No vertex buffer needed
vec3 position = proceduralPosition(gl_VertexID, u_time, u_params);

Upload: just uniforms (bytes). Geometry: computed.

IM4: Screen-Space Everything

If it can be computed in screen space, it's inherently immediate:

• SSAO computed fresh each frame
• Post-processing chains
• Raymarched volumes
• No geometry state, just pixels

IM5: Stateless Shader Uniforms

Instead of uploading mesh + transforms, upload:

• Procedural parameters
• Noise seeds
• Time values
• Mathematical definitions

Let shaders compute everything.

IM6: The "Fullscreen Quad + Math" Pattern

The most extreme immediate mode: render one quad, do everything in fragment shader.

// Entire 3D scene in fragment shader
void main() {
    vec3 ro = cameraPos;
    vec3 rd = normalize(rayDir(uv, cameraMatrix));
    
    float t = raymarch(ro, rd);
    vec3 p = ro + rd * t;
    vec3 n = calcNormal(p);
    vec3 col = shade(p, n, rd);
    
    fragColor = vec4(col, 1.0);
}

Data uploaded: Camera matrix, time, maybe a few floats. That's it.

IM7: Instanced Procedural Primitives

Draw N instances of nothing, generate everything from instance ID:

// "Mesh" is just instance count
// glDrawArraysInstanced(GL_TRIANGLES, 0, 36, 10000);  // 10000 cubes

void main() {
    int cubeID = gl_InstanceID;
    int vertexInCube = gl_VertexID;
    
    vec3 cubeCenter = proceduralPosition(cubeID, u_time);
    vec3 localPos = cubeVertex(vertexInCube);  // One of 36 vertices
    
    gl_Position = mvp * vec4(cubeCenter + localPos * scale, 1.0);
}

Uploaded data: Zero vertex buffers. Just uniforms and draw call.

IM8: Temporal Statelessness (No Frame Dependency)

Design rendering so frame N depends only on:

• Current time
• Current input state
• Constant parameters

Not on:

• Previous frame results
• Accumulated buffers
• History textures

This enables:

• Jump to any frame instantly
• Perfect reproducibility
• No temporal artifacts from past errors

IM9: Parameter Space Exploration

Because immediate mode recomputes everything, you can:

• Scrub time slider → see any moment
• Tweak any parameter → instant result
• No "rebuild" or "rebake" step

This is why ShaderToy is so effective for exploration.

IM10: Debug Injection Points

In immediate mode, you can inject debug visualization anywhere:

vec3 color = shade(p, n);

#ifdef DEBUG_NORMALS
    color = n * 0.5 + 0.5;
#endif

#ifdef DEBUG_DEPTH
    color = vec3(t / maxDist);
#endif

#ifdef DEBUG_ITERATIONS
    color = vec3(float(iterations) / float(MAX_STEPS));
#endif

No retained state to corrupt. Recompile shader, see debug view.

Immediate Mode Performance Patterns

Since immediate mode pays costs every frame, specific optimization patterns emerge:

IMP1: Hierarchical Early-Out

Instead of retaining a BVH, compute hierarchical bounds analytically:

float sceneSDF(vec3 p) {
    // Coarse bounds check (almost free)
    if (length(p - sceneCenter) > sceneRadius) return length(p - sceneCenter) - sceneRadius;
    
    // Check sub-regions
    float d = MAX_DIST;
    for (int region = 0; region < 8; region++) {
        vec3 regionCenter = ...;
        if (length(p - regionCenter) < regionRadius + d) {
            d = min(d, regionSDF(p, region));
        }
    }
    return d;
}

IMP2: Level of Detail via Iteration Count

In ray marching, distance = natural LOD:

// Far objects: fewer iterations automatically (larger steps)
// Near objects: more iterations (smaller steps near surface)
// LOD is emergent, not stored

IMP3: Resolution as LOD

Render at lower resolution, upscale. Immediate mode makes this trivial:

// Half res: every frame
// Full res: every frame
// No retained buffers to manage between modes

IMP4: Shader Complexity Budget

Fixed time budget per frame. In immediate mode, you control this directly:

if (technique_too_slow) {
    reduce MAX_STEPS;
    // or simplify SDF
    // or reduce resolution
    // Immediate feedback on cost
}

The Immediate Mode Development Loop

1. Write/modify shader code
2. Save file
3. Hot-reload shader (< 100ms)
4. See result
5. Repeat

Total iteration time: seconds

Compare to retained mode:

1. Modify scene graph code
2. Recompile application
3. Restart application
4. Wait for assets to load
5. Navigate to test location
6. See result
7. Repeat

Total iteration time: minutes

When Retained Mode is Actually Necessary

Despite the benefits of immediate mode, retained is necessary when:

1. Asset complexity exceeds real-time generation
   • Photogrammetry scans
   • Artist-authored detail
   • Motion capture data
2. Performance requirements demand it
   • 60+ fps with complex scenes
   • Mobile power constraints
   • VR latency requirements
3. Tooling expects it
   • Content pipelines
   • Level editors
   • Team workflows
4. Network multiplayer
   • Shared state must be synchronized
   • Can't recompute from different random seeds

The key insight: Start immediate, retain when proven necessary.

Interactive Shader Development & Debugging

The current state of shader debugging is abysmal. You write code, compile, run, see wrong output, and then... stare at the code? Add return vec4(debugValue, 0, 0, 1) and recompile? This section explores better approaches.

The Problem with Current Shader Debugging

Current workflow:
1. Write shader
2. Compile (hope for no errors)
3. Run application
4. See wrong output
5. Guess what's wrong
6. Add debug output (return color = normal, etc.)
7. Recompile entire application
8. Navigate back to test case
9. Squint at colors trying to interpret values
10. Repeat 5-9 approximately 47 times
11. Finally find the bug
12. Remove debug code
13. Discover you introduced a new bug while debugging

Total time: hours

What We Actually Want: Shader REPL/Notebook

Imagine a development environment where:

┌─────────────────────────────────────────────────────────────────┐
│ SHADER NOTEBOOK                                          [Run] │
├─────────────────────────────────────────────────────────────────┤
│ Cell 1: Define SDF primitives                                   │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ (define (sphere p r)                                        │ │
│ │   (- (length p) r))                                         │ │
│ │                                                             │ │
│ │ (define (box p b)                                           │ │
│ │   (let ((q (- (abs p) b)))                                  │ │
│ │     (+ (length (max q 0)) (min (max q.x (max q.y q.z)) 0))))│ │
│ └─────────────────────────────────────────────────────────────┘ │
│ [✓ Compiled] [Visualize] [Trace]                                │
│                                                                 │
│ Cell 2: Compose scene                                           │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ (define (scene p)                                           │ │
│ │   (smooth-union                                              │ │
│ │     (sphere (- p (vec3 0 1 0)) 0.5)                         │ │
│ │     (box p (vec3 1 0.2 1))                                  │ │
│ │     0.3))                                                   │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ [✓ Compiled] [Visualize] [Trace]                                │
│                                                                 │
│ ┌──────────────────────┐  ┌─────────────────────────────────┐  │
│ │   [Live Preview]     │  │ Trace at cursor (123, 456):     │  │
│ │                      │  │   p = (0.23, 1.02, -0.5)        │  │
│ │     ████████         │  │   sphere_d = 0.021              │  │
│ │   ██████████████     │  │   box_d = 0.83                  │  │
│ │   ██████████████     │  │   smooth_union_d = 0.019        │  │
│ │     ████████         │  │   iterations = 34               │  │
│ │                      │  │   total_distance = 4.23         │  │
│ └──────────────────────┘  └─────────────────────────────────┘  │
│                                                                 │
│ REPL> (scene (vec3 0 1 0))                                      │
│ => 0.0                                                          │
│ REPL> (gradient scene (vec3 0 1 0))                             │
│ => (0.0, 1.0, 0.0)  ; normal points up, correct                 │
│ REPL> _                                                         │
└─────────────────────────────────────────────────────────────────┘

DEB1: Pixel-Level Trace Capture

Concept: Click on any pixel, get full execution trace for that pixel.

Implementation approaches:

1. Debug shader variant: Compile shader with trace buffer writes

   #ifdef TRACE_ENABLED
   traceBuffer[traceIndex++] = vec4(TRACE_ID_SPHERE_DIST, d, 0, 0);
   #endif

2. Single-pixel compute dispatch: Re-run shader for just clicked pixel with full tracing
3. Printf debugging (where supported):

   // Vulkan debugPrintfEXT, requires validation layers
   debugPrintfEXT("p=%v3f d=%f", p, d);

Works well with:

• Immediate mode rendering (stateless, reproducible)
• Ray marching (single pixel = single ray = traceable)
• Compute shaders (can capture arbitrary data)

Does not work well with:

• Retained mode (which state version?)
• Geometry shaders (amplification complicates tracing)
• Multi-pass (which pass?)

DEB2: Hot-Reload with State Preservation

Concept: Edit shader, see result instantly, without losing camera position or scene state.

Implementation:

File watcher detects change
  → Compile new shader (background thread)
  → If success: swap shader, keep all uniforms
  → If fail: show error inline, keep old shader running
  → Never restart application

Existing tools with this:

• Bonzomatic (demoscene live coding)
• ShaderToy (web, but loses state on error)
• glslViewer (command line, file watching)
• Synthclipse (Eclipse-based)

What's missing: Integration with actual game engine scenes.

DEB3: In-Editor Visualization HUDs

Concept: Overlay debug info directly on the rendered frame, togglable via hotkeys or REPL.

┌────────────────────────────────────────────────────────┐
│ [F1] Normals  [F2] Depth  [F3] Iterations  [F4] UVs   │
│ [F5] Overdraw [F6] Mip Level [F7] Shader Cost         │
├────────────────────────────────────────────────────────┤
│                                                        │
│   Scene renders here with selected overlay             │
│                                                        │
│   ┌─────────────────────┐                              │
│   │ Stats:              │                              │
│   │ Avg iterations: 42  │                              │
│   │ Max iterations: 128 │                              │
│   │ Pixels at max: 2.3% │                              │
│   │ Frame time: 4.2ms   │                              │
│   └─────────────────────┘                              │
│                                                        │
│ > _                               [REPL input here]    │
└────────────────────────────────────────────────────────┘

REPL commands:

> set maxIterations 64          ; tweak uniform
> reload                        ; hot reload shaders
> capture                       ; save trace for this frame
> show normals                  ; enable normal vis
> query 320 240                 ; trace specific pixel
> time scene                    ; profile scene SDF
> bisect scene p0 p1 0.001      ; find exact surface point

DEB4: Value Inspection via Color Mapping

Since GPUs output colors, encode values as colors systematically:

// Standard debug palette
vec3 debugValue(float v, float minV, float maxV) {
    float t = (v - minV) / (maxV - minV);
    // Turbo colormap or similar
    return turboColormap(t);
}

// Directional encoding
vec3 debugDirection(vec3 d) {
    return d * 0.5 + 0.5;  // [-1,1] -> [0,1]
}

// Signed distance encoding  
vec3 debugSDF(float d) {
    if (d < 0.0) return vec3(-d, 0, 0);  // Inside: red
    else return vec3(0, 0, d);            // Outside: blue
}

Enhancement: Configurable color mapping with legend overlay.

DEB5: Time-Travel Debugging

For immediate mode rendering, we can replay any frame:

Record: [Camera, Time, Uniforms] for each frame
Playback: Jump to any recorded frame, re-render, trace

Timeline:
[====|====|====|====|====|====|====|====]
                    ^ Frame 847
                      Bug appears here
                      
[Step Back] [Step Forward] [Slow Motion] [Trace This Frame]

Works extremely well with: Immediate mode (stateless) Difficult with: Retained mode (need to record all state changes)

DEB6: Differential Rendering

Compare two shader versions side-by-side or as diff:

┌──────────────────────────────────────────────────────────────┐
│ [Before]              [After]               [Diff]           │
│ ┌────────────────┐    ┌────────────────┐   ┌────────────────┐│
│ │                │    │                │   │                ││
│ │  Old shader    │    │  New shader    │   │  Difference    ││
│ │  output        │    │  output        │   │  (amplified)   ││
│ │                │    │                │   │                ││
│ └────────────────┘    └────────────────┘   └────────────────┘│
│                                                              │
│ Max diff: 0.023 at (234, 567)                                │
│ Mean diff: 0.0001                                            │
│ Pixels > threshold: 0.3%                                     │
└──────────────────────────────────────────────────────────────┘

Useful for:

• Optimization (verify output unchanged)
• Bug hunting (when did it break?)
• Precision debugging (float issues)

DEB7: Shader Profiling Integration

Per-pixel cost visualization:

// Instrument shader to count operations
#ifdef PROFILE_MODE
    int opCount = 0;
    #define SIN(x) (opCount++, sin(x))
    #define SQRT(x) (opCount++, sqrt(x))
    // etc.
#endif

// Or use hardware counters where available
// AMD: shader_clock, shader_cycles
// NVIDIA: similar with NSight

Overlay: Heatmap of shader cost per pixel.

S-Expression Based Shader Language

The verbosity and inflexibility of GLSL/HLSL actively impedes the immediate-mode, exploratory workflow. A Lisp-like language would enable:

1. Macros - Generate repetitive code, DSLs within the language
2. Homoiconicity - Code as data, enabling powerful metaprogramming
3. Conciseness - Less boilerplate, more signal
4. REPL-friendly - Natural fit for interactive development

SEXPR1: Core Language Design

; Define an SDF primitive
(define (sphere p r)
  (- (length p) r))

; Define a combinator (this is where macros shine)
(defmacro smooth-union (a b k)
  `(let ((h (clamp (+ 0.5 (/ (- ,b ,a) ,k)) 0.0 1.0)))
     (- (mix ,b ,a h) (* ,k h (- 1.0 h)))))

; Use it
(define (scene p)
  (smooth-union
    (sphere p 1.0)
    (sphere (- p (vec3 1.5 0 0)) 0.8)
    0.3))

; Domain repetition as a macro
(defmacro repeat-domain (p spacing body)
  `(let ((,p (- (mod (+ ,p (* ,spacing 0.5)) ,spacing) (* ,spacing 0.5))))
     ,body))

; Infinite spheres!
(define (infinite-spheres p)
  (repeat-domain p (vec3 3.0)
    (sphere p 0.5)))

SEXPR2: Compilation Targets

Source (.scm/.lisp)
       │
       ▼
   ┌───────────┐
   │  Parser   │
   └───────────┘
       │
       ▼
   ┌───────────┐
   │   Macro   │ ◄── User-defined macros expand here
   │ Expansion │
   └───────────┘
       │
       ▼
   ┌───────────┐
   │    IR     │ ◄── S-expression intermediate representation
   └───────────┘
       │
       ├──────────────┬──────────────┬──────────────┐
       ▼              ▼              ▼              ▼
   ┌───────┐     ┌───────┐     ┌───────┐     ┌───────┐
   │ GLSL  │     │ HLSL  │     │ SPIR-V│     │ Metal │
   │  Gen  │     │  Gen  │     │  Gen  │     │  Gen  │
   └───────┘     └───────┘     └───────┘     └───────┘
       │              │              │              │
       ▼              ▼              ▼              ▼
   OpenGL         DirectX        Vulkan        macOS/iOS

SEXPR3: Example Macro Power

Problem: GLSL has no generics, no templates, lots of repetition.

GLSL approach:

float smoothUnionFloat(float a, float b, float k) { ... }
vec2 smoothUnionVec2(vec2 a, vec2 b, float k) { ... }
vec3 smoothUnionVec3(vec3 a, vec3 b, float k) { ... }
vec4 smoothUnionVec4(vec4 a, vec4 b, float k) { ... }

S-expr approach:

(defmacro define-smooth-union (type)
  `(define (,(symbol-append 'smooth-union- type) a b k)
     (let ((h (clamp (+ 0.5 (/ (- b a) k)) 0.0 1.0)))
       (- (mix b a h) (* k h (- 1.0 h))))))

(define-smooth-union float)
(define-smooth-union vec2)
(define-smooth-union vec3)
(define-smooth-union vec4)

; Or even better, let type inference handle it:
(define-generic smooth-union (a b k)
  (let ((h (clamp (+ 0.5 (/ (- b a) k)) 0.0 1.0)))
    (- (mix b a h) (* k h (- 1.0 h)))))

SEXPR4: Domain-Specific Extensions

; SDF-specific syntax
(sdf-define scene
  (union
    (intersect
      (sphere origin 1.0)
      (plane (vec3 0 1 0) 0.0))
    (transform (translate 0 2 0)
      (rotate-y time
        (box origin (vec3 0.5))))))

; Expands to efficient GLSL with proper transforms

SEXPR5: Debugging Integration

; Trace macro for debugging
(defmacro trace (label expr)
  (if *debug-mode*
    `(let ((result ,expr))
       (emit-trace ,label result)
       result)
    expr))

; Usage
(define (scene p)
  (trace 'final-distance
    (smooth-union
      (trace 'sphere-distance (sphere p 1.0))
      (trace 'box-distance (box p (vec3 1)))
      0.3)))

; In release mode, traces compile away completely

SEXPR6: Existing Work & Tools

Existing Lisp-to-shader compilers:

• Varjo (Common Lisp → GLSL, mature, used in production)
• Shady (Scheme-like → GLSL)
• CEPL (Common Lisp live-coding environment with Varjo)
• Shader-lib (Racket → GLSL)
• Penumbra (Clojure → GLSL, older)

Related approaches:

• ISF (Interactive Shader Format) - JSON metadata + GLSL
• The Book of Shaders Editor - Live preview but still GLSL
• KodeLife - Commercial live-coding tool
• cables.gl - Visual node-based, compiles to GLSL

What's missing: Deep integration with game engines, not just standalone tools.

SEXPR7: Minimal Kernel Language Approach

Alternative: Define a tiny core, sugar on top:

Core primitives (the "kernel"):
- lambda, let, if, define
- Arithmetic: + - * / mod
- Comparisons: < > = 
- Vector ops: vec2, vec3, vec4, swizzle
- Math: sin, cos, sqrt, abs, min, max, clamp, mix, length, dot, cross, normalize

Everything else: macros in user space

This kernel compiles directly to GLSL/HLSL/SPIR-V intrinsics.

SEXPR8: The REPL Experience

shader-repl> (define (sphere p r) (- (length p) r))
; sphere defined

shader-repl> (sphere (vec3 0 0 0) 1.0)
; => -1.0  (inside the sphere)

shader-repl> (sphere (vec3 2 0 0) 1.0)
; => 1.0  (outside the sphere)

shader-repl> (gradient sphere (vec3 1 0 0) 0.001)
; => (1.0, 0.0, 0.0)  (normal points +X)

shader-repl> (render scene #:width 800 #:height 600)
; [Opens preview window]

shader-repl> (set! *camera-pos* (vec3 0 2 5))
; [Preview updates immediately]

shader-repl> (export-glsl scene "scene.glsl")
; Wrote scene.glsl

shader-repl> (time (scene (vec3 0 0 0)))
; 0.000023ms per evaluation (CPU reference)
; Estimated GPU: ~0.0001ms per pixel

Engine Compatibility Notes

The unfortunate reality: most techniques must eventually work with Godot, Unreal, or Unity. Here's a realistic assessment of integration difficulty.

Compatibility Rating Scale

★★★★★ - Native support, just use it
★★★★☆ - Plugin/extension exists, works well  
★★★☆☆ - Possible with effort, some limitations
★★☆☆☆ - Hacky, fights the engine, fragile
★☆☆☆☆ - Theoretically possible, practically painful
☆☆☆☆☆ - Fundamentally incompatible, don't try

Engine Architecture Summary

Technique Compatibility Matrix

Geometry Techniques

Procedural Techniques

Texture Techniques

Rendering Paradigms

Animation

Immediate Mode & Debugging

Integration Strategies by Engine

Godot 4 Integration

Advantages:

• Open source - can modify anything
• GDExtension for native code
• Simple, readable codebase
• Shader language is clean and well-documented
• Best hot-reload support

Challenges:

• Smaller ecosystem
• Less mature advanced features
• Compute shader support newer

Recommended approach for immediate-mode/SDF:

1. Create GDExtension in C++/Rust
2. Write custom VisualServer rendering commands
3. Use ComputeShader for generation
4. Hook into Godot's shader preprocessor for custom syntax

Difficulty: ★★★☆☆ (Medium, source helps)

S-expr shader integration:

Option A: Transpile to Godot Shading Language
  - Godot's shader lang is simple, good target
  - Loses some GLSL features but gains simplicity
  
Option B: GDExtension that compiles to SPIR-V directly
  - Bypass Godot's shader compiler
  - More complex but more powerful

Difficulty: ★★★☆☆ (Medium)

Unreal Engine 5 Integration

Advantages:

• Extremely powerful rendering features
• Good compute shader support
• Nanite, Lumen, Virtual Texturing built-in
• Large community, many tutorials

Challenges:

• Massive, complex codebase
• Strong opinions about how things should be done
• Material Editor is visual, not text-friendly
• Custom rendering is very complex
• Shader compilation is slow

Recommended approach for immediate-mode/SDF:

1. Use Custom Stencil Buffer approach (hacky)
2. Or: Create custom SceneViewExtension
3. Or: Modify Deferred Shading pass (requires engine mod)
4. For simple cases: Ray march in post-process material

Difficulty: ★★☆☆☆ (Hard, fights the engine)

S-expr shader integration:

Option A: Generate HLSL, inject via Custom node in Materials
  - Limited to material graph context
  - Can't easily do full custom passes
  
Option B: Generate USF files, register as global shaders
  - More powerful, much more complex
  - Need to understand RDG (Render Dependency Graph)
  
Option C: Plugin that preprocesses .usf files
  - Intercept shader compilation
  - Very fragile, version-dependent

Difficulty: ★☆☆☆☆ (Very Hard)

What UE5 is good at (use these instead):

• Nanite for massive geometry - just use it
• Lumen for GI - just use it
• Material Editor for standard PBR - accept the visual workflow
• Niagara for particles/procedural

Unity Integration

Advantages:

• SRP (Scriptable Render Pipeline) allows custom rendering
• C# is reasonable for tooling
• Good shader variant system
• Moderate complexity (between Godot and Unreal)
• Shader preprocessing possible

Challenges:

• Closed source
• SRP documentation incomplete
• Multiple render pipelines (URP vs HDRP) fragment ecosystem
• Shader compilation can be slow

Recommended approach for immediate-mode/SDF:

1. Create custom RenderFeature in URP/HDRP
2. Add custom render pass
3. Full control over what's rendered
4. Can inject compute shaders

Difficulty: ★★★☆☆ (Medium, SRP helps)

S-expr shader integration:

Option A: Transpile to HLSL, use as .shader files
  - Most straightforward
  - Can use ShaderLab wrapper for multi-platform
  
Option B: Custom ScriptedImporter for .scm files
  - Editor recognizes your file type
  - Compiles on import
  - Good workflow integration

Option C: Shader preprocessing via IPreprocessShaders
  - Intercept compilation
  - Transform AST before Unity compiles
  
Difficulty: ★★★☆☆ (Medium, good extensibility)

Recommended Engine by Technique Priority

If your priority is...

The "Escape Hatch" Approach

When engines fight you too much:

Strategy: Render to texture in custom code, composite in engine

┌─────────────────────────────────────────────────────────────┐
│                      Your Custom Renderer                    │
│                                                             │
│  ┌─────────────────┐     ┌─────────────────┐               │
│  │  S-expr Shader  │ ──► │  GLSL/HLSL/MSL  │               │
│  │    Compiler     │     │                 │               │
│  └─────────────────┘     └─────────────────┘               │
│           │                       │                         │
│           ▼                       ▼                         │
│  ┌─────────────────────────────────────────┐               │
│  │        Custom Rendering Context          │               │
│  │   (Separate window or shared context)    │               │
│  └─────────────────────────────────────────┘               │
│                         │                                   │
│                         ▼                                   │
│                  Render to Texture                          │
│                         │                                   │
└─────────────────────────┼───────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                    Game Engine                               │
│                                                             │
│   ┌───────────────────────────────────────────────────┐     │
│   │  Apply texture to quad / full-screen pass          │     │
│   │  Engine handles: UI, physics, audio, input, etc.   │     │
│   └───────────────────────────────────────────────────┘     │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Benefits:

• Full control over rendering
• Engine handles everything else
• Clean separation of concerns
• Can develop renderer independently

Drawbacks:

• Context switching overhead
• Synchronization complexity
• Lose some engine integration (shadows, etc.)
• Two codebases to maintain

Future: WebGPU as Universal Target?

WebGPU is emerging as a potential universal shader target:

S-expr source
     │
     ▼
  WGSL output
     │
     ├──► Browser (native WebGPU)
     ├──► Desktop (Dawn/wgpu)
     └──► Engines (via plugins)
         ├──► Godot (wgpu backend in development)
         ├──► Unity (WebGPU export)
         └──► Unreal (less likely)

Advantages:

• Single target language (WGSL)
• Modern API design
• Cross-platform by design
• Good for immediate-mode patterns

Current status: Promising but not production-ready for games.

Spectrum Position Affects Compression Strategy

Key insight: In immediate mode, your "compression" is your procedural algorithm. The SDF function IS the compressed representation of your geometry.

Mapping Other Techniques to Spectrum

Observed Patterns & Principles

After cataloging these techniques, several meta-patterns emerge:

Pattern 1: Rightward Dependency (Immediate Can Use Retained, Not Vice Versa)

RETAINED ◄─────────────────────────────────── IMMEDIATE
   D1    D2    D3    D4    D5    D6    D7    D8    D9    D10
   
   ────────────────────────────────────────────────────────►
   Easy to depend on things to the RIGHT (more immediate)
   
   ◄────────────────────────────────────────────────────────
   Hard to depend on things to the LEFT (more retained)

Why this happens:

• Immediate mode code can always be "frozen" and retained
• Retained mode code assumes data exists; can't easily generate it on demand
• You can cache the output of procedural generation, but you can't easily proceduralize cached data

Examples:

• A retained mesh (D2) can easily use a procedural texture (D8) - just compute it once, bake to texture
• A procedural SDF (D8) cannot easily use a pre-built mesh (D2) - would need SDF conversion
• Nanite (D1) can't invoke compute-generated geometry (D8) - it needs pre-clustered data

Implication: Design your most constrained/retained systems first, then fill in with more immediate techniques.

Pattern 2: Compression Relevance Inversely Correlates with Dynamism

High Compression Value ◄─────────────────► Low Compression Value
        D1    D2    D3    D4    D5    D6    D7    D8    D9    D10
        
The more retained, the more compression matters.
The more immediate, the less traditional compression applies.

At D1-D3: Heavy compression (Draco, BC7, Basis) - amortized over long retention At D4-D6: Moderate compression - balance decode speed vs. size At D7-D8: Light/fast compression - streaming, decode cost matters At D9-D10: Algorithm IS compression - procedural = maximally compressed

Implication: If you're targeting extreme compression, you're implicitly pushed toward immediate/procedural.

Pattern 3: Debugging Ease Correlates with Dynamism

Hard to Debug ◄───────────────────────────► Easy to Debug
     D1    D2    D3    D4    D5    D6    D7    D8    D9    D10

Why:

• D1: State on GPU, async, no visibility
• D5: Mixed, can focus on one half
• D10: Everything in CPU memory, print statements work

Implication: Develop in immediate mode (D8-D10), then migrate leftward for performance.

Pattern 4: Engine Compatibility Inversely Correlates with Dynamism

Good Engine Support ◄────────────────────► Poor Engine Support
      D1    D2    D3    D4    D5    D6    D7    D8    D9    D10
      
(With exception of D1 in UE5 specifically due to Nanite)

Why: Game engines are built around retained scene graphs (D2-D3). They have pipelines for:

• Loading meshes → GPU buffers
• Managing transforms
• Culling, batching, draw calls

They don't have pipelines for:

• "Here's a function, evaluate it per-pixel"
• "Generate geometry in this compute shader, render it, discard"

Exception: UE5's Nanite is D1, but required rewriting Unreal's entire renderer.

Implication: The more immediate your technique, the more you fight the engine.

Pattern 5: The "Goldilocks Zone" is D5-D6

For most practical game development:

• D1-D2: AAA with dedicated engine teams
• D3-D4: Standard production games
• D5-D6: Best balance of flexibility and compatibility
• D7-D8: Specialized (voxel games, demoscene)
• D9-D10: Prototyping, learning, extreme cases

D5-D6 characteristics:

• CPU understands what's being rendered
• Can use both retained and immediate techniques
• Debuggable
• Works with engines (with some effort)
• Reasonable performance

Pattern 6: Procedural Techniques Cluster at D8

Almost all procedural techniques (P1-P8) naturally live at D8:

• They compute rather than store
• Output can be captured (→ move left) or computed fresh (→ stay)
• Parameters are tiny (bytes), output is large (megabytes)

The procedural bargain: Trade compute for storage.

Pattern 7: Style Constraints Enable Extreme Positions

The ART* techniques (single primitive, single palette, low resolution) enable:

• D10 becomes viable (CPU can handle simple rendering)
• D8-D9 become practical (less to compute)
• Compression becomes trivial (less variety = better ratios)

Implication: Artistic constraints are technical enablers.

Pattern 8: Animation Breaks Immediate Mode Assumptions

Animation is the hardest category for immediate mode because:

• Keyframe data must come from somewhere (artists)
• Skeletal hierarchies need state
• Blending requires history

Solutions:

• A4 (Procedural animation) - stays immediate
• A5 (VAT) - bake to texture, become retained
• Accept hybrid: retained animation data, immediate rendering

Pattern 9: The Two Cultures

There are essentially two graphics programming cultures:

Culture A: Retained/Engine (D1-D4)

• "Load asset, add to scene, engine handles it"
• Performance via caching, batching, culling
• Debug via engine tools
• Artist-driven content

Culture B: Immediate/Procedural (D7-D10)

• "Compute everything from parameters"
• Performance via algorithm efficiency
• Debug via printf and visualization
• Programmer-driven content

Most techniques belong clearly to one culture. Mixing them is possible but requires bridging.

Pattern 10: The Demoscene Position

Demoscene 4KB/64KB intros occupy D8-D10 exclusively:

• No room for retained data
• Everything procedural
• Executable packer as final compression
• SDF + ray marching dominant

This represents the theoretical minimum for 3D graphics: pure algorithm, zero assets.

Technique × Spectrum Compatibility Matrix

This table shows which techniques work at which spectrum positions.

Legend:

• ● = Native/natural fit
• ◐ = Possible with effort
• ○ = Theoretically possible but awkward
• ✗ = Incompatible/nonsensical

Geometry Techniques

(Additional technique × spectrum tables for Procedural, Texture, Rendering, etc. continue in the full document)

Technique × Technique Compatibility Matrix

These matrices show how different techniques work together.

Legend:

• ✅ = Strong synergy (designed to work together)
• ⚠️ = Possible (can combine with effort)
• ❌ = Incompatible (fundamentally different paradigms)
• ➖ = Same technique / not applicable

Core Geometry Techniques

Key Observations:

• G2 (SDF) is fundamentally incompatible with polygon-based techniques (G3, G6, G9)
• G6 (Nanite) works great with traditional meshes, impostors for LOD
• G4 (Voxel) bridges SDF and mesh worlds - can convert to either
• G8 (Noise) is highly compatible with procedural approaches (SDF, voxel, heightmap)

Geometry × Rendering Paradigms

Key Observations:

• R2 (Ray Marching) is the natural match for SDF, incompatible with most mesh techniques
• G6 (Nanite) uses software rasterization internally, integrates with deferred
• R7 (Ray Tracing) works with traditional geometry, not pure SDF or procedural

Geometry × Texture Techniques

Key Observations:

• T6 (Procedural Textures) is the natural match for SDF and procedural geometry
• T7 (Triplanar) is essential for geometry without UV coordinates (voxels, SDF, CSG)
• G5 (Splats) have their own color representation, don't use traditional texturing
• T1 (Compression) doesn't apply to pure procedural (nothing to compress)

Geometry × Animation

Key Observations:

• A2 (Skeletal) only works with deformable mesh representations (G3, G9)
• A4 (Procedural) is the natural match for SDF and procedural geometry
• G6 (Nanite) has limited skeletal support (world-space evaluation issue)

Geometry × Lighting

Key Observations:

• M4 (Lightmaps) requires UV-unwrapped static geometry - incompatible with SDF, noise, CSG
• M1 (Flat) and M7 (Toon) work with almost everything
• G3 (Mesh) is the most lighting-compatible representation

Mega-Matrix: Core Techniques (Condensed)

A condensed view of the most important technique interactions across all categories:

Paradigm Clusters Visible:

• Top-left (G2, P6, T6, R2, A4): Procedural/Immediate cluster - lots of ✅
• Center-right (G3, G6, R5, M3, M4, A2): Traditional/Retained cluster - lots of ✅
• Cross-paradigm (❌ patterns): SDF↔Mesh, Lightmap↔Procedural, RayMarch↔Deferred

Fundamental Paradigm Splits

The matrix reveals four fundamental splits:

Compatibility Recipes

Pre-validated technique combinations for common use cases:

Recipe: Demoscene 4KB

G2 (SDF) + P8 (Fractal) + T6 (Procedural) + R2 (RayMarch) + M2 (Phong) + A4 (Procedural)

✅ All compatible - pure procedural, no stored data

Recipe: Mobile Game

G3 (Mesh) + G1 (Billboard) + T1 (Compress) + T3 (Atlas) + R1 (Forward) + M4 (Lightmap) + A2 (Skeletal) + L1 (Discrete LOD)

✅ All compatible - traditional retained, baked lighting

Recipe: Open World AAA

G6 (Nanite) + G3 (Mesh) + P2 (Terrain) + T4 (Virtual) + R5 (Deferred) + R7 (RT) + M3 (PBR) + M4 (Lightmap) + A2 (Skeletal) + L5 (Streaming)

✅ All compatible - Nanite + traditional with streaming

Recipe: Voxel Sandbox

G4 (Voxel) + P6 (Compute) + T8 (Palette) + R3 (VoxelCast) + M2 (Phong) + A4 (Procedural) + C7 (DAG)

✅ All compatible - voxel-focused stack

Recipe: Stylized Indie

G3 (Mesh) + G9 (Subdiv) + T6 (Procedural) + T8 (Palette) + R1 (Forward) + M7 (Toon) + M8 (Matcap) + A2 (Skeletal) + A4 (Procedural)

✅ All compatible - stylized with procedural touches

Procedural Techniques

Texture Techniques

Rendering Paradigms

Compression & Encoding

LOD & Streaming

Animation

Lighting & Materials

Artistic Constraints

Immediate Mode Techniques

Reading the Matrix

To find techniques for a specific spectrum position: Read down the column. ● and ◐ are viable choices.

To see how universal a technique is: Read across the row. More ● = more universal.

Most universal techniques (work everywhere):

• C2: Quantization
• M1: Flat/Unlit
• M2: Lambert/Phong
• M7: Toon Shading
• M8: Matcaps
• ART2: Single Palette
• ART3: Low Resolution
• ART5: Silhouette Design
• R8: Software Rasterization (ironic but true)

Most position-specific techniques:

• G6: Nanite (D1 only)
• M4: Lightmaps (D1-D3 only)
• G2: SDFs (D5-D10 only)
• R2: Ray Marching (D5-D10 only)

Reference Appendix

This appendix provides search-based links to documentation, examples, papers, and tools. Using search links rather than direct URLs helps prevent link rot.

Search Link Format

All links use DuckDuckGo searches. Format: https://duckduckgo.com/?q=<search+terms>

Demoscene References

4KB/64KB Intros (Extreme Procedural)

• Search: pouet.net 4k intro procedural
• Search: iq shadertoy 4kb demoscene techniques
• Search: crinkler exe packer demoscene
• Search: kkrunchy shader minifier
• Search: Elevated by RGBA Conspiracy 4k breakdown
• Search: demoscene procedural texture generation

Demoscene Tools

• Search: Shadertoy SDF tutorial
• Search: Bonzomatic live coding shader
• Search: GNU Rocket sync tracker demoscene
• Search: Werkkzeug procedural tool demoscene

Signed Distance Fields (G2, R2)

Tutorials & Fundamentals

• Search: Inigo Quilez SDF functions
• Search: "hg_sdf" GLSL library
• Search: ray marching signed distance field tutorial
• Search: sphere tracing John Hart paper

Advanced SDF Techniques

• Search: SDF smooth minimum blend shapes
• Search: SDF ambient occlusion cheap
• Search: SDF soft shadows ray marching
• Search: "Enhanced Sphere Tracing" paper

SDF in Games

• Search: Dreams PS4 SDF rendering Media Molecule
• Search: Claybook voxel SDF game

Voxels (G4)

Algorithms

• Search: sparse voxel octree GPU rendering
• Search: sparse voxel DAG compression gigavoxels
• Search: marching cubes algorithm implementation
• Search: dual contouring sharp features voxel
• Search: transvoxel algorithm LOD

Voxel Games/Engines

• Search: Teardown voxel engine breakdown
• Search: Minecraft chunk rendering optimization
• Search: Atomontage voxel engine

Nanite / Virtualized Geometry (G6)

• Search: UE5 Nanite technical deep dive
• Search: Nanite cluster culling software rasterizer
• Search: "A Deep Dive into Nanite" GDC Unreal
• Search: GPU-driven rendering pipeline visibility buffer
• Search: meshlet GPU culling mesh shaders

Gaussian Splatting / NeRF (G5, H2)

• Search: 3D Gaussian Splatting real-time paper
• Search: NeRF neural radiance fields original paper
• Search: instant-ngp NVIDIA fast NeRF
• Search: gaussian splatting Unity plugin
• Search: Luma AI gaussian splat

Procedural Generation (P1-P8)

Buildings & Architecture

• Search: procedural building generation L-system
• Search: Wave Function Collapse algorithm
• Search: shape grammar architecture procedural
• Search: Houdini procedural building tutorial

Terrain

• Search: hydraulic erosion simulation terrain
• Search: fractal terrain generation fBm
• Search: Sebastian Lague terrain generation
• Search: GPU terrain tessellation CDLOD

Vegetation

• Search: L-system tree generation algorithm
• Search: space colonization algorithm tree
• Search: SpeedTree procedural

Noise Functions

• Search: Perlin noise implementation GPU
• Search: Simplex noise Ken Perlin
• Search: Worley noise cellular
• Search: curl noise fluid simulation

Texture Compression (T1-T16)

GPU Formats

• Search: BC7 texture compression quality
• Search: ASTC texture compression mobile
• Search: Basis Universal texture supercompression
• Search: KTX2 texture format specification

Procedural Textures

• Search: procedural texture GLSL noise
• Search: reaction diffusion texture shader
• Search: "The Book of Shaders" noise

Virtual Texturing

• Search: virtual texturing megatexture implementation
• Search: UE5 virtual texture streaming

Mesh Compression (G7)

• Search: Draco mesh compression Google
• Search: meshoptimizer vertex cache
• Search: glTF mesh compression extension
• Search: OpenCTM mesh compression

Animation Compression (A3, A5)

• Search: ACL animation compression library
• Search: vertex animation texture Houdini
• Search: skeletal animation quaternion compression
• Search: motion matching animation

Compute Shader Generation (P6, D8)

• Search: compute shader mesh generation GPU
• Search: GPU marching cubes compute shader
• Search: indirect draw compute shader culling
• Search: mesh shaders amplification geometry

S-Expression / Lisp Shader Languages (SEXPR)

• Search: Varjo Common Lisp GLSL compiler
• Search: CEPL Common Lisp live coding OpenGL
• Search: Racket shader language graphics
• Search: "Shady" Scheme to GLSL
• Search: GLSL transpiler compiler implementation

Shader Debugging & Development (DEB)

• Search: RenderDoc GPU debugger tutorial
• Search: NVIDIA Nsight Graphics shader debugging
• Search: PIX shader debugging DirectX
• Search: shader hot reload implementation
• Search: glslViewer file watching live
• Search: Vulkan debugPrintfEXT shader printf

Engine Documentation

Godot

• Search: Godot 4 custom shader documentation
• Search: Godot GDExtension rendering
• Search: Godot compute shader tutorial
• Search: Godot RenderingServer custom draw
• Search: site:docs.godotengine.org shading language

Unreal Engine

• Search: UE5 custom render pass plugin
• Search: Unreal Engine global shaders USF
• Search: UE5 RDG Render Dependency Graph
• Search: Unreal compute shader tutorial
• Search: site:docs.unrealengine.com rendering

Unity

• Search: Unity SRP Scriptable Render Pipeline custom
• Search: Unity custom render feature URP
• Search: Unity compute shader tutorial
• Search: Unity ScriptedImporter custom asset
• Search: site:docs.unity3d.com shader

Research Papers

Rendering

• Search: "Real-Time Rendering" book Akenine-Moller
• Search: GPU Gems chapters online NVIDIA
• Search: SIGGRAPH real-time rendering course

SDFs & Ray Marching

• Search: "Sphere Tracing" paper 1996 John Hart
• Search: "Ray Marching and Signed Distance Functions"

Voxels

• Search: "GigaVoxels" paper GPU voxel
• Search: "Efficient Sparse Voxel Octrees" paper
• Search: "High Resolution Sparse Voxel DAGs"

Compression

• Search: "Geometry Compression" Deering paper
• Search: progressive mesh Hoppe paper

Procedural

• Search: "Texturing and Modeling: A Procedural Approach" book
• Search: "Real-time Procedural Generation" survey

WebGPU / WGSL

• Search: WebGPU specification W3C
• Search: WGSL shader language specification
• Search: wgpu Rust WebGPU implementation
• Search: Dawn WebGPU Chrome
• Search: WebGPU tutorial getting started

Audio (Bonus)

• Search: 4klang software synthesizer demoscene
• Search: Clinkster synth demoscene
• Search: tracker music MOD XM format
• Search: procedural audio synthesis game

Code Size Reduction

• Search: Crinkler executable compressor
• Search: kkrunchy exe packer
• Search: shader minifier GLSL tool
• Search: UPX executable packer
• Search: squishy executable compression

Notable Talks & Presentations

• Search: "Rendering of Inside" GDC Playdead
• Search: "Low Complexity, High Fidelity" GDC INSIDE
• Search: Media Molecule Dreams GDC sculpting SDF
• Search: "Practical Real-Time Strategies for Accurate Indirect Occlusion"
• Search: "Advances in Real-Time Rendering" SIGGRAPH course

Community Resources

• Search: pouet.net demoscene productions
• Search: Shadertoy popular shaders
• Search: reddit r/proceduralgeneration
• Search: reddit r/graphicsprogramming
• Search: graphics programming discord
• Search: demoscene discord community

Hardware & Software Requirements

This section catalogs the minimum hardware and API requirements for each technique, enabling filtering by target platform.

Requirement Tiers

Hardware Tiers

Mobile Hardware Tiers

Software/API Tiers

Spectrum Positions: Requirements

Technique Requirements Matrix

Geometry Techniques

Procedural Generation Techniques

Texture Techniques

Rendering Paradigms

Compression & Encoding

LOD & Streaming

Animation Techniques

Lighting & Materials

Shader/Immediate Mode Techniques

Debugging & Development

API Feature Requirements Detail

OpenGL / OpenGL ES Version Mapping

Mobile-Specific Considerations

Platform Compatibility Quick Reference

"Works Everywhere" Techniques (HW0/MW0 + SW0)

These techniques work on virtually any GPU from the last 20 years:

Geometry:

• G1: Billboards
• G3: Polygon Meshes
• G7: Mesh Compression (CPU decode)
• G11: CSG (mesh booleans)
• G13: Height Maps
• G14: Sprite Stacking

Textures:

• T2: Basis Universal (transcodes)
• T3: Atlasing
• T7: Triplanar Mapping
• T8: Palette-Based
• T12: Detail Textures
• T14: Material ID

Rendering:

• R1: Forward Rendering
• R4: Particles (CPU)
• R9: Lightmaps/PRT

Animation:

• A1: Keyframe
• A2: Skeletal (CPU skinning)
• A6: Morph Targets
• A8: Flipbook

Lighting:

• M1: Flat/Unlit
• M2: Lambert/Phong
• M4: Lightmaps
• M7: Toon Shading
• M8: Matcaps

LOD:

• L1: Discrete LOD
• L3: HLOD
• L4: Impostor LOD
• L5: Streaming

"Modern Baseline" Techniques (HW2/MW2 + SW2)

These require compute shaders and are the current mainstream:

Geometry:

• G4: Voxels (compute generation)
• G5: Gaussian Splats
• G9: Subdivision (GPU tessellation)
• G10: Displacement (GPU)

Procedural:

• P1: Proc Buildings (GPU)
• P2: Proc Terrain (GPU erosion)
• P6: Compute Generation

Textures:

• T4: Virtual Texturing
• T11: Neural Compression

Rendering:

• R4: Particles (GPU compute)
• R6: Forward+
• R8: Software Rasterization

Compression:

• C7: Sparse Voxel DAG (GPU)

"Cutting Edge" Techniques (HW4+ / SW4)

These require latest hardware:

Geometry:

• G6: Nanite-style (mesh shaders help)

Rendering:

• R7: Hybrid Ray Tracing

Features used:

• Mesh shaders
• Hardware ray tracing
• Variable rate shading
• Advanced async compute

WebGL Compatibility Notes

WebGL has specific limitations that affect technique viability:

*Extensions may provide some features

WebGL 1.0 Compatible Techniques:

• All "Works Everywhere" techniques
• Basic SDF ray marching (IM1, R2)
• Procedural textures (P4, T6)
• Simple noise functions

WebGL 2.0 Additional:

• 3D textures (baked SDFs)
• Instancing (IM7)
• Transform feedback
• MRT (deferred shading)

Requires WebGPU:

• Any compute-based generation
• Modern particle systems
• GPU-driven rendering

VR-Specific Requirements

VR adds additional constraints:

VR-Friendly:

• G6 (Nanite) - geometry efficient
• R1 (Forward) - low latency
• M4 (Lightmaps) - cheap GI
• L1-L4 (LOD) - essential

VR-Challenging:

• R2 (Ray marching) - 2x cost for stereo
• R5 (Deferred) - bandwidth on mobile VR
• Heavy post-processing

Recommended Minimum Targets by Project Type

Interactive Filter Criteria (for future website)

The website should allow filtering by:

Hardware:

• Ancient (HW0/MW0) - 2006 and earlier
• Old (HW1/MW1) - 2006-2012
• Moderate (HW2/MW2) - 2012-2018
• Modern (HW3/MW3) - 2018-2022
• Current (HW4/MW4) - 2022+
• Cutting Edge (HW5) - Latest features

Platform:

• Desktop (Windows/Mac/Linux)
• Mobile (iOS/Android)
• Web (WebGL 1.0)
• Web (WebGL 2.0)
• Web (WebGPU)
• VR (Standalone)
• VR (PC-tethered)
• Console (current gen)
• Console (last gen)

API:

• OpenGL 2.1 / ES 2.0 / DX9
• OpenGL 3.3 / ES 3.0 / DX10
• OpenGL 4.3 / ES 3.1 / DX11
• Vulkan / DX12 / Metal
• Vulkan 1.3 / DX12 Ultimate / Metal 3

Required Features:

• Compute shaders
• Tessellation
• Geometry shaders
• Mesh shaders
• Ray tracing
• Bindless textures
• Indirect draw
• 3D textures

Spectrum Position:

• D1-D3 (Retained)
• D4-D6 (Mixed)
• D7-D10 (Immediate)

When user selects constraints, techniques that don't meet requirements are grayed out or hidden, and the compatibility matrix updates to show only viable combinations.

Offline / Online & Interactivity

This section addresses the temporal dimension of when computation happens and how mutable/interactive the results can be.

The Two Axes

                        INTERACTIVITY
                             │
            Static ◄─────────┼─────────► Fully Interactive
         (view only)         │         (modify anything)
                             │
    ─────────────────────────┼─────────────────────────────
                             │
         Offline ◄───────────┼───────────► Online
       (precompute)     COMPUTATION      (realtime)
                          TIMING

Computation Timing (Offline ↔ Online):

• Offline: Computed before runtime (build time, bake time, preprocessing)
• Online: Computed during runtime (per-frame, on-demand)

Interactivity (Static ↔ Interactive):

• Static: Cannot be modified at runtime (view/playback only)
• Interactive: Can be modified by user/gameplay in real-time

Key insight: Online enables interactive, but doesn't guarantee it. You can compute something every frame that the user can't meaningfully change.

Computation Timing Spectrum

Interactivity Levels

The Precomputation Trade-off

More Precomputation ───────────────────────► Less Precomputation
                                            
    ┌─────────────────────────────────────────────────────────┐
    │ OFFLINE                          │ ONLINE               │
    │                                  │                      │
    │ • Higher quality possible        │ • Lower quality      │
    │ • More computation time          │ • Strict time budget │
    │ • Results are static             │ • Results can change │
    │ • Storage cost for results       │ • Compute cost       │
    │ • Iteration requires rebake      │ • Instant iteration  │
    │ • Works on weak runtime HW       │ • Needs strong HW    │
    └─────────────────────────────────────────────────────────┘

The fundamental trade-off:

• Precompute more → Higher quality, less interactive
• Precompute less → Lower quality (or higher HW req), more interactive

Precomputation Techniques Catalog

PRE1: Texture Mipmapping

Description: Pre-generate filtered lower-resolution versions of textures.

Computation Timing: CT0 (build-time) or CT1 (load-time) Interactivity Impact: None - transparent optimization Storage Cost: +33% texture size Runtime Benefit: Proper filtering, reduced bandwidth, fewer artifacts

What it enables:

• Correct minification filtering
• Anisotropic filtering quality
• Reduced aliasing
• Texture streaming (load lower mips first)

Without it: Runtime would need to filter on-the-fly (expensive) or accept aliasing.

Works with: All texture-based techniques Engine support: Universal, automatic

PRE2: Lightmap Baking

Description: Pre-compute global illumination, shadows, ambient occlusion into textures.

Computation Timing: CT0 (build-time), minutes to hours Interactivity Impact: I0-I1 (lighting is static, can only move camera/dynamic objects) Storage Cost: Significant (often largest asset) Runtime Benefit: Complex GI "for free" at runtime

What it enables:

• Path-traced quality GI
• Soft shadows from area lights
• Multiple bounce illumination
• Works on mobile/low-end

Without it: Need runtime GI (expensive) or accept flat lighting.

Variants:

• Directional lightmaps (store dominant direction)
• Radiosity normal mapping (store for multiple directions)
• Spherical harmonic lightmaps

Limitations:

• Static geometry only
• UV unwrapping required
• Long bake times
• Large storage

Works with: M4, R9, any static environment Does not work with: Fully dynamic scenes, destructible geometry

PRE3: LOD Generation

Description: Pre-generate simplified mesh versions for distance rendering.

Computation Timing: CT0 (build-time) Interactivity Impact: None - transparent optimization Storage Cost: ~50-100% additional mesh data Runtime Benefit: Massive triangle reduction at distance

Approaches:

• Mesh decimation (Simplygon, MeshLab)
• Manual artist LODs
• Automatic edge collapse
• Impostor baking (billboard from mesh)

What it enables:

• Draw millions of objects at distance
• Consistent frame rate across view distances
• Lower memory bandwidth

Works with: L1, L3, L4, standard mesh workflows Does not work with: Fully procedural geometry (must regenerate or cache)

PRE4: Shader Compilation / Permutation Baking

Description: Pre-compile shader variants for all material/feature combinations.

Computation Timing: CT0 (build-time) or CT1 (load-time) Interactivity Impact: None - transparent Storage Cost: Can be huge (permutation explosion) Runtime Benefit: No shader compilation stalls

Problem it solves: Runtime shader compilation causes frame hitches.

Approaches:

• Compile all permutations offline
• Shader warmup passes at load time
• Pipeline state object caching (Vulkan/DX12)
• Shader precompilation (Metal)

Challenges:

• Permutation explosion (N features → 2^N shaders)
• Platform-specific compilation
• Update requires rebuild

PRE5: Navigation Mesh Generation

Description: Pre-compute walkable surface representation for AI pathfinding.

Computation Timing: CT0 (build-time) Interactivity Impact: I0-I3 (navmesh is static, but objects can move on it) Storage Cost: Small-moderate Runtime Benefit: Fast pathfinding

What it enables:

• A* pathfinding at scale
• AI navigation
• Crowd simulation

Dynamic alternatives:

• Runtime navmesh generation (CT3) - expensive but allows destruction
• Hierarchical pathfinding
• Flow fields

PRE6: Physics Collision Mesh Simplification

Description: Pre-generate simplified collision geometry from render mesh.

Computation Timing: CT0 (build-time) Interactivity Impact: Physics is interactive (I3), but collision shape is static Storage Cost: Small Runtime Benefit: Fast collision detection

Approaches:

• Convex hull decomposition
• Simplified proxy meshes
• Primitive fitting (boxes, capsules, spheres)
• Bounding volume hierarchies

PRE7: Ambient Occlusion Baking

Description: Pre-compute per-vertex or texture-space ambient occlusion.

Computation Timing: CT0 (build-time) Interactivity Impact: I0-I1 (AO is static) Storage Cost: Per-vertex: small. Texture: moderate Runtime Benefit: High-quality AO without SSAO cost

Variants:

• Vertex AO (per-vertex float)
• Texture AO (dedicated channel)
• Bent normal baking (AO direction)

Runtime alternative: SSAO, GTAO, HBAO (CT3)

PRE8: SDF Volume Baking

Description: Pre-compute signed distance field in 3D texture from mesh.

Computation Timing: CT0 (build-time), can be slow Interactivity Impact: I0-I1 (SDF is static, but enables dynamic queries) Storage Cost: Moderate-high (3D texture) Runtime Benefit: Fast distance queries, soft shadows, AO

What it enables:

• Mesh → SDF conversion for hybrid rendering
• Soft shadows from SDF
• Global distance field (UE5)
• Collision detection acceleration

Tools:

• SDFGen, mesh-to-sdf utilities
• Raytraced SDF baking
• Jump flooding algorithm

Works with: G2 (as alternative to analytical SDF), H5 (hybrid)

PRE9: Visibility / PVS Baking

Description: Pre-compute which areas can see which other areas.

Computation Timing: CT0 (build-time), can be hours Interactivity Impact: I0-I1 (visibility is static) Storage Cost: Moderate Runtime Benefit: Instant occlusion culling

What it enables:

• Skip rendering of guaranteed-hidden objects
• Reduce draw calls dramatically in interiors
• Enable streaming decisions

Classic approach: Potentially Visible Sets (Quake-era) Modern: GPU occlusion queries, hierarchical Z-buffer (online)

Works with: Indoor environments, levels with walls Does not work with: Open worlds, destructible environments

PRE10: Texture Compression

Description: Pre-compress textures to GPU-friendly formats.

Computation Timing: CT0 (build-time) Interactivity Impact: None - transparent Storage Cost: Reduced (that's the point) Runtime Benefit: Lower memory, faster sampling

Formats: BC1-7, ASTC, ETC1/2 (see T1)

Why offline: Quality compression (BC7, ASTC) is slow. Runtime transcoding (Basis) is faster but still CT1.

PRE11: Animation Compression

Description: Pre-compress animation curves, removing redundancy.

Computation Timing: CT0 (build-time) Interactivity Impact: None - animation plays back identically Storage Cost: Reduced Runtime Benefit: Lower memory, potentially faster playback

Techniques:

• Curve fitting
• Keyframe reduction
• Quantization
• Delta encoding
• ACL library

PRE12: Impostor / Billboard Baking

Description: Pre-render 3D object from multiple angles to use as billboard.

Computation Timing: CT0 (build-time) Interactivity Impact: I0-I1 (impostor is static representation) Storage Cost: Texture atlas per object Runtime Benefit: Render complex objects as single quad

Variants:

• Simple billboard (1 view)
• Octahedral impostor (8-24 views, interpolate)
• Spherical impostor (full sphere of views)

What it enables:

• Massive forests, crowds at distance
• Complex objects as LOD fallback

Works with: L4, G1

PRE13: Convolution / Cubemap Preprocessing

Description: Pre-filter environment maps for different roughness levels.

Computation Timing: CT0 or CT1 Interactivity Impact: I0-I1 (environment is static) Storage Cost: Moderate (mip chain of cubemaps) Runtime Benefit: PBR specular IBL at single texture sample

What it enables:

• Split-sum IBL approximation
• Roughness-correct reflections
• Single-sample specular

Without it: Would need importance sampling at runtime (expensive).

Related: Pre-integrated BRDF LUT

PRE14: Mesh Optimization / Vertex Cache Optimization

Description: Pre-reorder vertices and indices for GPU cache efficiency.

Computation Timing: CT0 (build-time) Interactivity Impact: None - transparent Storage Cost: None (same data, different order) Runtime Benefit: Better GPU cache utilization, faster rendering

Tools: meshoptimizer, AMD Tootle, Forsyth algorithm

Should always do this: Free performance, no downside.

PRE15: Occlusion Data Baking (Per-Object)

Description: Pre-compute occlusion contribution of objects for culling hints.

Computation Timing: CT0 (build-time) Interactivity Impact: I0-I3 (objects can move, but their occlusion properties are static) Storage Cost: Small per object Runtime Benefit: Better culling decisions

Example: Mark objects as "large occluder" or "never occludes"

PRE16: Terrain Erosion Simulation

Description: Pre-simulate hydraulic/thermal erosion on heightmap terrain.

Computation Timing: CT0 (build-time), minutes Interactivity Impact: I0-I1 (terrain is static) Storage Cost: Just heightmap Runtime Benefit: Realistic terrain without runtime simulation

Can be online: GPU erosion simulation exists but expensive.

Works with: P2, G13

PRE17: Procedural Content Caching / Baking

Description: Run procedural generation offline, store results.

Computation Timing: CT0 (build-time) or CT1 (load-time, seed-based) Interactivity Impact: I0-I2 (can't modify the generation, only parameters) Storage Cost: Full generated content Runtime Benefit: Use procedural algorithms that are too slow for real-time

Pattern:

Offline: Run expensive procedural algorithm
         Store result as static asset
Online:  Load and use as normal mesh/texture

Enables: Complex generation algorithms (expensive erosion, multi-pass synthesis) Loses: Runtime variation, interactivity

PRE18: Cluster / Meshlet Generation

Description: Pre-divide meshes into GPU-friendly clusters for culling.

Computation Timing: CT0 (build-time) Interactivity Impact: None - transparent Storage Cost: Additional index data Runtime Benefit: Enables per-cluster culling, mesh shaders

Required for: G6 (Nanite-style), mesh shader pipelines Tools: meshoptimizer, Nanite importer

PRE19: BVH / Acceleration Structure Building

Description: Pre-build bounding volume hierarchies for ray tracing or collision.

Computation Timing: CT0 (static) or CT3 (dynamic, for RT) Interactivity Impact: Varies - static BVH = I0, dynamic rebuild = I3+ Storage Cost: Moderate Runtime Benefit: Fast ray-geometry intersection

For ray tracing: Modern RT APIs build/update AS at runtime, but static geometry can pre-build.

PRE20: Voxelization

Description: Pre-convert mesh to voxel representation.

Computation Timing: CT0 (build-time) Interactivity Impact: I0-I1 (voxels are static) Storage Cost: Depends on resolution Runtime Benefit: Enables voxel-based algorithms (GI, AO)

What it enables:

• Voxel cone tracing GI
• SDF from voxels
• Collision acceleration

Technique Classification: Offline / Online / Interactivity

Geometry Techniques

Procedural Techniques

Texture Techniques

Rendering Paradigms

Lighting & Materials

Animation

Precomputation in the Demoscene

Demoscene has a unique relationship with precomputation because executable size is limited:

What demoscene precomputes (at build time):

• Shader minification
• Executable packing
• Music pattern data
• Nothing else (no room for baked data!)

What demoscene computes at runtime:

• All geometry (SDFs, procedural)
• All textures (procedural noise, patterns)
• All animation (procedural, sync to music)
• Everything else

The demoscene philosophy: Precomputation requires storage. Extreme size limits push everything online. The "compression" is the algorithm itself.

Research insight: Demoscene techniques prove what's achievable with zero precomputation, establishing the lower bound of quality at any given runtime budget.

Precomputation Strategy by Project Type

Interactive Filter Additions (for website)

Add these filters to the interactive website:

Computation Timing:

• Build-time only (CT0)
• Load-time acceptable (CT0-CT1)
• Streaming/background OK (CT0-CT2)
• Must be per-frame (CT3+)
• Must be per-pixel/procedural (CT4)

Interactivity Required:

• Static is fine (I0)
• View-only interactive (I1)
• Parameter tweaking (I2)
• Object manipulation (I3)
• Content editing (I4)
• Full generative (I5)

Precomputation Constraints:

• No storage for precomputed data (demoscene)
• Limited storage (mobile)
• Moderate storage OK (PC indie)
• Unlimited storage (AAA)

Cross-Reference: Precomputation × Hardware

Some precomputation can offload runtime requirements:

Pattern: Heavy precomputation can enable running on weaker hardware by trading storage for compute.

Cross-Reference: Precomputation × Spectrum Position

Geometry Representation

G1: Billboards / Impostors

Description: Replace 3D geometry with camera-facing 2D planes textured with pre-rendered or stylized images.

Variants:

• Simple billboards (always face camera)
• Axial billboards (rotate around one axis, e.g., trees)
• Octahedral impostors (8-12 pre-rendered views, blend between)
• Spherical impostors (full sphere of views)

Works well with:

• G15 (Style transfer post-processing)
• T3 (Texture atlasing)
• T8 (Palette-based textures)
• R4 (Particle systems)
• L2 (Distance-based LOD)

Does not work well with:

• R2 (Ray marching SDFs) - fundamentally different paradigm
• G4 (Voxels) - conflicting representations
• A2 (Skeletal animation) - billboards lack skeletal structure
• Lighting that requires normal information (unless using normal maps)

G2: Signed Distance Fields (SDFs)

Description: Represent geometry as mathematical functions returning distance to nearest surface. Rendered via ray marching.

Variants:

• Analytical SDFs (pure math functions)
• Baked SDF textures (3D textures storing distances)
• Hybrid (analytical + baked details)

Works well with:

• P1 (Procedural geometry via CSG operations)
• S1 (Domain repetition for instancing)
• S5 (Smooth blending between shapes)
• R2 (Ray marching renderer)
• T7 (Triplanar mapping)
• A4 (Procedural animation)

Does not work well with:

• G3 (Traditional polygon meshes) - different pipeline
• G6 (Nanite-style virtualized geometry)
• Hardware tessellation
• Traditional rasterization pipeline
• Existing game engine workflows

Engine Compatibility:

• Godot: ★★★☆☆ - Custom shader + render pass, doable with GDExtension
• Unreal: ★★☆☆☆ - Post-process material (limited) or engine mod (hard)
• Unity: ★★★☆☆ - Custom RenderFeature in SRP, reasonable

Spectrum Position: D8-D10 (inherently immediate mode)

G3: Traditional Polygon Meshes (Baseline)

Description: Standard triangle/quad meshes. Included for compatibility reference.

Works well with:

• Most standard techniques
• G6 (Nanite instancing)
• A1, A2, A3 (Standard animation)
• All standard texture techniques

Does not work well with:

• R2 (Ray marching) - requires conversion
• G2 (SDFs) - different representation

G4: Voxels

Description: Represent geometry as 3D grids of filled/empty cells, optionally with material data.

Variants:

• Dense voxel grids
• Sparse voxel octrees (SVO)
• Sparse voxel DAGs (directed acyclic graphs - massive compression)
• Voxel run-length encoding

Works well with:

• P2 (Procedural terrain/caves)
• C3 (Octree compression)
• R3 (Voxel ray casting)
• T8 (Palette-based materials)
• Destructible environments
• L3 (Octree-based LOD)

Does not work well with:

• G1 (Billboards) - conflicting paradigms
• Smooth organic shapes (without very high resolution)
• A2 (Skeletal animation) - voxels don't deform well
• Traditional GPU rasterization (needs conversion)

G5: Point Clouds / Splats

Description: Represent surfaces as collections of points, rendered as small discs or gaussians.

Variants:

• Raw point clouds
• Gaussian splats (3D Gaussian Splatting)
• Surfels (surface elements with normals)

Works well with:

• Photogrammetry/scanned data
• L2 (Distance-based LOD via point density)
• Neural compression techniques
• Real-time capture scenarios

Does not work well with:

• Sharp edges and hard surfaces
• A2 (Skeletal animation)
• Traditional texture mapping
• Small file sizes (typically large datasets)

G6: Virtualized Geometry (Nanite-style)

Description: Stream and render massive meshes by breaking into clusters, culling aggressively, and using software rasterization for small triangles.

Works well with:

• G3 (Traditional meshes as source)
• Massive instance counts
• L1 (Seamless LOD)
• Highly detailed static environments

Does not work well with:

• G2 (SDFs)
• G4 (Voxels)
• A2 (Skeletal animation) - limited support
• Very low memory budgets (needs streaming infrastructure)
• Procedural geometry (needs pre-built clusters)

Engine Compatibility:

• Godot: ☆☆☆☆☆ - Would require rewriting renderer
• Unreal: ★★★★★ - Native, this is UE5's flagship feature
• Unity: ☆☆☆☆☆ - No equivalent, would need custom implementation

Spectrum Position: D1 (maximally retained, GPU-driven)

G7: Mesh Compression Formats

Description: Compress traditional meshes using specialized algorithms.

Variants:

• Draco (Google) - aggressive vertex/index compression
• Meshoptimizer - vertex cache optimization + compression
• OpenCTM
• Corto
• Quantization (reduce vertex precision)

Works well with:

• G3 (Traditional meshes)
• Any mesh-based workflow
• Network streaming

Does not work well with:

• Procedural geometry (already minimal)
• G2, G4 (Different representations)

G8: Implicit Surfaces via Noise Functions

Description: Define surfaces as isosurfaces of noise functions (e.g., Perlin, Simplex, Worley).

Works well with:

• P2 (Procedural terrain)
• G2 (SDFs) - noise as SDF modifier
• Marching cubes extraction
• T6 (Procedural textures)

Does not work well with:

• Precise authored shapes
• Character models
• Architectural details

G9: Catmull-Clark / Subdivision Surfaces

Description: Store coarse control cage, subdivide at runtime to desired detail.

Works well with:

• Hardware tessellation
• L1 (Adaptive LOD based on screen coverage)
• Animation (animate control cage only)
• Organic shapes

Does not work well with:

• Hard surface details
• Very low-end hardware
• Real-time LOD transitions (popping)

G10: Displacement Mapping on Simple Base Meshes

Description: Use simple base geometry + displacement maps for detail.

Works well with:

• G9 (Subdivision surfaces)
• Hardware tessellation
• T1 (Texture compression)
• Terrain rendering

Does not work well with:

• Extreme displacement (silhouette issues)
• Very low poly bases
• Mobile/low-end hardware

G11: Constructive Solid Geometry (CSG)

Description: Build complex shapes from boolean operations on primitives.

Works well with:

• G2 (SDFs) - trivial CSG with min/max operations
• P1 (Procedural buildings)
• Architectural/mechanical shapes
• Level editors

Does not work well with:

• Organic shapes
• Traditional mesh pipeline (needs baking)
• Real-time modification at scale

G12: Curve-Based Geometry (Bezier, B-Splines, NURBS)

Description: Define surfaces mathematically via control points and basis functions.

Works well with:

• CAD-like precision
• Tessellation at runtime
• Vehicle/architectural modeling
• Fonts and text

Does not work well with:

• Organic characters
• Texture mapping (parameterization issues)
• Game engine standard workflows

G13: Height Maps (2.5D Geometry)

Description: Represent terrain as 2D grid of heights.

Works well with:

• P2 (Procedural terrain)
• L2 (Distance-based LOD like CDLOD, geoclipmaps)
• T1 (Standard texture compression)
• Large outdoor environments

Does not work well with:

• Overhangs, caves, arches
• Vertical surfaces
• Indoor environments

G14: Sprite Stacking

Description: Stack 2D sprite layers to create pseudo-3D effect (popular in indie games).

Works well with:

• 2D art pipelines
• Pixel art aesthetics
• Simple rotation
• Low memory budgets

Does not work well with:

• Free camera rotation (limited angles)
• Smooth surfaces
• Complex lighting

G15: Style Transfer / Neural Rendering on Billboards

Description: Render simple geometry, apply neural style transfer to add apparent detail/dimension.

Works well with:

• G1 (Billboards)
• Artistic/stylized games
• Post-processing pipeline

Does not work well with:

• Realistic graphics
• Consistent frame-to-frame appearance (temporal stability)
• Low-end hardware (neural networks expensive)

G16: Geometry Images

Description: Encode mesh geometry as 2D images (positions, normals as RGB). GPU-friendly compression.

Works well with:

• Standard image compression
• GPU texture sampling
• Streaming

Does not work well with:

• High genus topology
• Sharp features
• Very detailed meshes

G17: Tetrahedral Meshes

Description: Represent solid volumes with tetrahedra for simulation/deformation.

Works well with:

• Soft body physics
• Destruction simulation
• Medical/scientific viz

Does not work well with:

• Standard rendering (need surface extraction)
• Real-time at high detail
• Typical game assets

Procedural Generation

P1: Procedural Building/Structure Generation

Description: Generate architectural geometry from rules, grammars, or parameters.

Variants:

• L-systems
• Shape grammars
• Wave Function Collapse
• Houdini-style procedural graphs

Works well with:

• G2 (SDFs) for component shapes
• G11 (CSG) for combining
• T6 (Procedural textures)
• Infinite/large worlds

Does not work well with:

• Hand-authored unique buildings
• Very specific architectural requirements
• Interiors with complex layouts

P2: Procedural Terrain

Description: Generate terrain from noise functions, erosion simulation, etc.

Variants:

• Fractal noise (fBm, ridged multifractal)
• Erosion simulation (hydraulic, thermal)
• Tectonic simulation
• Machine learning terrain

Works well with:

• G13 (Height maps)
• G8 (Noise-based implicit surfaces)
• T6 (Procedural textures)
• L2 (Terrain LOD systems)

Does not work well with:

• Specific authored terrain features
• Multiplayer synchronization (seed matching)
• Playtesting consistency

P3: Procedural Vegetation

Description: Generate trees, plants from L-systems or space colonization algorithms.

Works well with:

• G1 (Billboard LOD for distance)
• Wind animation
• Large forests with variety

Does not work well with:

• Specific tree shapes
• Very low memory (still need some parameters)

P4: Procedural Textures (Runtime)

Description: Generate textures entirely in shaders from noise and math.

Works well with:

• G2 (SDFs) - same mathematical mindset
• Infinite detail at any resolution
• Zero texture memory

Does not work well with:

• Specific authored looks
• Photorealistic materials
• Caching/performance on low-end

P5: Wang Tiles / Corner Tiles

Description: Small set of tiles that can be combined infinitely without visible repetition.

Works well with: