Shader Language ECS, Part 1 - Idea

Exploring WGSL compute shaders to implement ECS state management concepts

I'll explore a simple way to bring the Entity Component System (ECS) pattern into the world of WGSL shaders.

This is still in idea stage, an idea that i'll explore in this post without providing a real implementation, just the API for it.

While I have no particular goal in mind with this I enjoy exploring new technology (WGSL) and discover what it can do for me. I'm writing this post to share my findings.

"A cat with two wheels on his eyes"

Can we leverage the GPU for more than visuals?

This is not a new concept, Danil has made a cool exposition and demo of his approach, with a shadertoy demo of it.

ECS - Entity Component System

ECS is a simple idea that brings detachment to the app state and embraces sparse relationships. It's not my purpose to give an intro to ECS in this article, if you want to learn more about ECS check this wikipedia page and the many yt videos about it."

The TLDR of an Entity Component System, is that Entities, Components and Systems are concepts that you as a programmer define, but in a way that by default they are unrelated to each other.

Imagine three arrays: one for entities, one for components, and one for systems. Each element exists in its own space, connected only through explicit references.

A car and a cat driving it, a list of wheels and a list of positions, cat and car share some of these attributes

So, ideally Entities and Components and Systems don't know much about each other:

Entities: Things in your game/app (e.g., a player, an enemy, a tree).
Components: The data or attributes of those things (e.g., position, health, color).
Systems: The logic that acts on and changes that data (e.g., movement, collision detection, AI).

The key is that systems work with any entity that has the right components, hopefully making it easy to add or change behavior without messing up the whole codebase.

Bringing ECS to the GPU

In WGSL you send a shader code string to the graphics api (WebGPU) which then compiles it into the GPU processor code that gets run with whatever settings you have set.

My idea is to make use of the shader code string creation and with JavaScript (JS) create hardcoded const arrays for the lookups and queries needed for each system.

i.e. If a system works on all entities that have a Moveable component and a Position component, then when creating the WGSL string two const arrays of numbers with the exact same size would be written in the string. These arrays are to be iterated sequentially in the system, and hold the index of the Position component and the index of the Moveable component in their respected storage buffers.

These static arrays represent the entities and components relationships and are set through JS, tailored each System needs. It's like using JS as a precompilation language with its Maps and Sets to perform the queries and dump the results in a WGSL string.

The key here is that the relationships between entities and components are defined upfront, in JS, before sending anything to the GPU. This way, the GPU only has to execute the pre-determined logic defined by CPU code (JS).

Ideally in the future someone could minimax this out to produce a highly optimized string ready to run on the GPU.

Attractiveness is sweet

WebGPU and WGSL are attractive because they have compute shaders, which are simpler versions of shaders for whatever you want.

Not only that, but unlike WebGL/GLSL, the new WGSL also brings support for pointers and compile time arrays ("const" arrays).

In WebGPU we can create our own pipeline, and dispatch a given entrypoint function of the shader with very specific parallel execution scenarios. I find this a cool thing and wanted to explore it a bit and see how it can be used to implement an ECS.

Compute shaders dispatches

The rough outline of the simplified ECS for WGSL is this:

Components are defined as WGSL structs/types.
JS is used to create the relationships between entities and components
Systems are defined as WGSL functions that read/update components
Systems function arguments define the components query they use
JS parses this system function and performs the queries.
The final WGSL string dispatches each system function once for each query result

The code outline

My objective is that in the end I can do something like this in JS and have a final WGSL shader string created:

const world = createWorld();

// Create an entity
const PLAYER = world.createEntity();

// Create a position component type (A simple 2d vector)
// and its initialization function (in WGSL)
const POSITION = world.createComponent(`
alias Position = vec2<f32>;

fn init_position(
  position: ptr<storage, Position, read_write>,
  i: u32,
  total_number_of_components: u32
) {
    let space = 2.0 / total_number_of_components;
    let left = -0.5;

    (*position).x = left + float(i) * space;
}

`);

// Create a distance movement type (here using a struct just because yes)
// No initialization this time. (in WGSL)
const MOVEABLE = world.createComponent(`
struct Moveable {
  delta: vec2f,
};
`);

// Relate these two components to the player
world.addComponentToEntity(PLAYER, POSITION);
world.addComponentToEntity(PLAYER, MOVEABLE);

// A movement system, defined using WGSL function
// This function will be called for all entities that
// have the Moveable and the Position components:
// (in WGSL)
world.createSystem(`
fn movement_system(
    moveable: ptr<storage, Moveable, read_write>,
    position: ptr<storage, Position, read_write>) {
    (*position).at += (*moveable).delta;
}
`);

// An input update system, same thing with a WGSL function
// that is called once for all entities that have the Moveable
// component.
world.createSystem(`
fn keyboard_system(moveable: ptr<storage, Moveable, read_write>) {
    const amount: f32 = 0.01;

    // Up:
    if (keyDown(38)) {
        (*moveable).delta.y = amount;
        (*moveable).delta.x = 0.0;
    } else
    // Down:
    if (keyDown(40)) {
        (*moveable).delta.y = -amount;
        (*moveable).delta.x = 0.0;
    } else
    // Left:
    if (keyDown(37)) {
        (*moveable).delta.x = -amount;
        (*moveable).delta.y = 0.0;
    } else
    // Right:
    if (keyDown(39)) {
        (*moveable).delta.x = amount;
        (*moveable).delta.y = 0.0;
    }
}
`);

// The shader string to send to the GPU
const shader = world.generateWGSL();

This is a rough idea for the API for now. It will evolve based on new findings and learnings I'll get along the way.

The output outline

I created this shader in compute.toys, as an example of code that I intend to implement as a WGSL string in this new API.

You can also play this video to see the shader in action.

Outlining it:

A WGSL string that does one dispatch per system function. With components hardcoded into it in static const arrays.
Initialization is done inline as much as possible.
Static component relationships. A dynamic update of a relationship would imply a string rewrite and WGSL shader recompilation.

The usecase for this API are shaders with entity component relationships that are mostly static. The bottleneck here comes from recompilation which is what I want to avoid in the first place.

Since WGSL recompilation is required whenever there's a dynamic update, this API is targetting shaders with static entity component relationships. Not to say that it can't handle dynamic updates but it will be at the cost of performance. For now my focus is on static relationships.

Open questions and next steps

I haven't measured the performance of the API so I can't list what are the pros and cons yet. However, I delined a few questions that will be my guide to substantiate a proper list of the cons and pros of this novel approach.

Is there some usefulness in having the state management done in the same language as the graphics?
- Could this lead to better low hanging fruit optimization opportunities and reduced communication overhead between CPU and GPU?
Will it be massively slow to re-compile a big shader with everything in it? (i would love to benchmark WGSL compilation times at some point)
- I think that this could be a major bottleneck, especially for dynamic scenes or frequent updates.
Can we easily produce mind-bending stuff when the state is tightly coupled with the graphics?
- There might be some potential here for unique visual effects and gameplay mechanics that make use of the GPU capabilities.
Debugging will be super hard (if not impossible), but I plan and hope to make some developments here in the near future.
Performance? I doubt there will be any performance gains at all immediately and I'm not sure I can produce good enough data points to conclude anything on this.

Conclusion

This post is just the first step in exploring this idea. I have no polished library or API to showcase yet – just the shader and API presented here at this stage.

My next step is to implement the API based on the example code and showcase its pros and cons.

In the next post, I'll attempt to bring this to life with a more concrete example, hopefully I'll have a bit more clarification on the potential benefits and drawbacks of it.

I'm both excited and apprehensive. Will this experiment crash and burn, or is there a hidden gem here somewhere?

Join me on this journey into the uncharted territory (for me at least) of ECS in WGSL!

Special thanks to Joana and António for the helpful edits and thoughtful suggestions