Shader Nodes

Motivations

jME3 material system is entirely based on shaders. While it’s pretty powerful, this system has some issues and limitations :

Monolithic shaders have a serious lack of flexibility, and it can be daunting to get into the code for inexperienced users.
Maintenance ease of such shaders is poor. (for example the whole lighting shaders represent around 500 lines of code, and it could be a lot worse with more features)
Adding new features to those shaders decrease the ease of maintenance a lot. This point made us reluctant to do so and some feature were never added (Fog to name it, but many more).
Users can’t add their own feature to the shader unless they fork it, and fall back to the same issues explained in previous points.

Shader Nodes were designed with this in mind and are the fruit of many long discussions in the core chat balancing the pros and cons of this or that pattern.
At first this system was referred to as “Shader” injection. The basic idea was to allow users to inject code into shaders with a tag replacement system.
We finally came with a different concept called Shader Nodes, that is inspired from blender nodes system for textures and post process.
The final shader is generated at run time by the system by assembling shader nodes together.

What is a Shader Node?

Conceptually, it’s just a self sufficient piece of glsl code that accepts inputs and produce some outputs.
Inputs are glsl variables that may be fed by previous nodes output values.
Outputs are glsl variables fed with values computed in the shader node code.

In practice it’s a bit more than that.A shader node is declined in several parts :

A shader node definition, defining :
- The type of shader node (Vertex or Fragment for now)
- The minimal glsl version needed for the shader node
- The path to the shader file (containing the shader code heh)
- A mandatory documentation block for this Shader node. As I hope many users will do their own nodes and contribute them back this point is crucial.
- A list of inputs accepted by this shader (typed glsl variable optional or needed for the code to run properly)
- A list of outputs produced by this shader (typed glsl variable computed and fed by the node’s code)
The actual shader code file (.vert or .frag depending on the node’s type like any shader file)
A shader node declaration having :
- A unique name(in the shader scope)
- The shader node definition it’s based on
- An optional activation condition (based on the actual define system)
- A list of input mapping (what will be actually fed to those inputs)
- A list of output mapping (what will be output to the global output of the shader)

Shader Node definition

First ShaderNodes have to be defined either in a separate file (j3sn for jme3 shader node) or directly embed in the Technique block of the j3md file.
Please refer to this documentation for global structure of a j3md file jMonkeyEngine3 Material Specification

All is included in a ShaderNodeDefinitions bloc. This block can have several nodes defined (it’s recommended to define nodes that have strong dependencies with each other in the same j3sn file).
A ShaderNode is declared in a ShaderNodeDefinition block.
The global structure should look like this :

ShaderNodeDefinitions{
      ShaderNodeDefinition <NodeDefName>{
            Type : <ShaderType>
            Shader <ShaderLangAndVersion> : <ShaderPath>
            [Shader <ShaderLangAndVersion> : <ShaderPath>]
            [...]
            Documentation {
                <DocContent>
            }
            Input {
                <GlslVarType> <VarName>
                [<GlslVarType> <VarName>]
                [...]
            }
            Output {
                <GlslVarType> <VarName>
                [<GlslVarType> <VarName>]
                [...]
            }
      }
      [ShaderNodeDefinition <NodeDef2Name> {
         [...]
      }]
      [...]
}

All that is not between [] is mandatory.

ShaderNodeDefinition : the definition block. You can have several definition in the same ShaderNodeDefinitions block.
- NodeDefName : The name of this ShaderNodeDefinition
Type : define the type of this shader node
- ShaderType : The type of shader for this definition. For now only “Vertex” and “Fragment” are supported.
Shader : the version and path of the shader code to use. note that you can have several shader with different GLSL version. The generator will pick the relevant one according to GPU capabilities.
- ShaderLangAndVersion : follows the same syntax than the shader declaration in the j3md file : GLSL<version>, version being 100 for glsl 1.0 , 130 for glsl 1.3, 150 for glsl 1.5 and so on. Note that this is the minimum glsl version this shader supports
- ShaderPath the path to the shader code file (relative to the asset folder)
Documentation : the documentation block. This is mandatory and I really recommend filling this if you want to contribute your shader nodes. This documentation will be read buy the SDK and presented to users willing to add this node to their material definitions. This should contain a brief description of the node and a description for each input and ouput.
- @input can be use to prefix an input name so the sdk recognize it and format it accordingly. the syntax id @input <inputName> <description>.
- @output can be use to prefix an output name so the sdk recognize it and format it accordingly. the syntax id @output <inputName> <description>
Input : The input block containing all the inputs of this node. A node can have 1 or several inputs.
- GlslVarType : a valid glsl variable type that will be used in the shader for this input. see https://www.khronos.org/opengl/wiki/GLSL_Type and the “Declare” an array chapter
- VarName : the name of the variable. Note that you can’t have several inputs with the same name.
Output : The output block containing all the outputs of this node. A node can have 1 or several outputs.
- GlslVarType : a valid glsl variable type that will be used in the shader for this input. see https://www.khronos.org/opengl/wiki/GLSL_Type and the “Declare” an array chapter
- VarName : the name of the variable. Note that you can’t have several outputs with the same name.

If you use the same name for an input and an ouput, the generator will consider them as the SAME variable so they should be of the same glsl type.

Example

Here is a typical shader node definition.

ShaderNodeDefinitions{
     ShaderNodeDefinition LightMapping{
        Type: Fragment
        Shader GLSL100: Common/MatDefs/ShaderNodes/LightMapping/lightMap.frag
        Documentation {
            This Node is responsible for multiplying a light mapping contribution to a given color.
            @input texCoord the texture coordinates to use for light mapping
            @input lightMap the texture to use for light mapping
            @input color the color the lightmap color will be multiplied to
            @output color the resulting color
        }
        Input{
            vec2 texCoord
            sampler2D lightMap
            vec4 color
        }
        Output{
            vec4 color
        }
    }
}

Declare an array

To declare an array you have to specify its size between square brackets.
Constant size
The size can be an int constant.
Example

      float myArray[10]

This will declare a float array with 10 elements. Any material parameter mapped with this array should be of FloatArray type and it’s size will be assumed as 10 when the shader is generated.

Material parameter driven size
The size can be dynamic and driven by a material parameter. GLSL does not support non constant values for array declaration so this material parameter will be mapped to a define.
Example

     float myArray[NumberOfElements]

This declares a float array with the size depending on the value of the NumberOfElement material parameter.
NumberOfElement HAS to be declared in the material definition as a material parameter. It will be mapped to a define and used in the shader.

If this value change the shader will have to be recompiled, due to the fact that it’s mapped to a define.

Shader Node code

The shader code associated with a Shader node is similar to any shader code.
The code for a Vertex shader node should be in a .vert file and the code for a Fragment shader node should be in a .frag file. It has a declarative part containing variable declaration, function declaration and so on… And a main part that is embed in a “void main(){}” block.
Input and output variables declared in the shader node definition can be used without being declared in the shader code. ( they shouldn’t even or you’ll have issues).
Here is a the code of the LightMap.frag shader.

void main(){
    color *= texture2D(lightMap, texCoord);
}

Very simple, it’s just a texture fetch, but of course anything can be done.
Do not declare uniforms, attributes or varyings in a shader node code, the Generator will handle this, just use the inputs and outputs and optional local variables you may need.

Shader Node declaration

To create a shader we need to plug shader nodes to each other, but also interact with built in glsl inputs and outputs. Shader nodes are declared inside the Technique block. The vertex nodes are declared in the VertexShaderNodes block and the fragment nodes are declared in the FragmentShaderNodes block.
Note that if the j3md has ember shader nodes definition (in a ShaderNodesDefinitions block) it must be declared before the VertexShaderNodes and FragmentShaderNodes blocks. Of course there can be several ShaderNode declaration in those block.
Here is how a ShaderNode declaration should look :

ShaderNode <ShaderNodeName>{
     Definition : <DefinitionName> [: <DefinitionPath>]
     [Condition : <ActivationCondition>]
     InputMapping{
          <InputVariableName>[.<Swizzle>] = <NameSpace>.<VarName>[.<Swizzle>] [: <MappingCondition>]
          [...]
     }
     [OutputMapping{
          <NameSpace>.<VarName>[.<Swizzle>] = <OutputVariableName>[.<Swizzle>] [: <MappingCondition>]
          [...]
     }]
}

ShaderNode the shader node block
- ShaderNodeName the name of this shader node, can be anything, but has to be unique in the shader scope.
Definition : a reference to the shader node definition.
- DefinitionName : the name of the definition this Node use. this definition can be declared in the same j3md or in its own j3sn file.
- DefinitionPath : in case the definition is declared in it’s own j3sn file, you have to set the path to this file here.
Condition a condition that dictates if the node is active or not.
- ActivationCondition : The condition for this node to be used. Today we use Defines to use different blocks of code used depending on the state of a Material Parameter. The condition here use the exact same paradigm. A valid condition must be the name of a material parameter or any combinations using logical operators “||,“&&”, “!” or grouping characters “(” and “)”. The generator will create the corresponding define and the shader node code will be embed into and #ifdef statement.

For example, let’s say we have a Color and ColorMap material parameter, this condition “Color || ColorMap” will generate this statement :

        #if defined(COLOR) || defined(COLORMAP)
            ...
        #endif

InputMapping the wiring of the inputs of this node, coming from previous node’s outputs or from built in glsl inputs.
- InputVariableName : the name of the variable to map as declared in the definition.
- Swizzle : Swizling for the preceding variable. More information on glsl swizzling on this page https://www.khronos.org/opengl/wiki/GLSL_Type.
- NameSpace : The generator will use variable name space to avoid collision between variable names. Name space can be one of these values :
  - MatParam : the following variable is a Material Parameter declared in the MaterialParameters block of the materialDefinition.
  - WorldParam : the following variable is a World Parameter declared in the WorldParameters block of the current technique block. World parameters can be one of those declared in this file : https://github.com/jMonkeyEngine/jmonkeyengine/blob/master/jme3-core/src/main/java/com/jme3/shader/UniformBinding.java
  - Attr : the following variable is a shader attribute. It can be one those declared in the Type enum of the VertexBuffer class https://github.com/jMonkeyEngine/jmonkeyengine/blob/master/jme3-core/src/main/java/com/jme3/scene/VertexBuffer.java.
  - Global : the variable is a global variable to the shader. Global variables will be assign at the end of the shader to glsl built in outputs : gl_Position for the vertex shader, or to one of the possible outputs of the fragment shader (for example gl_FragColor). The global variable can have what ever name pleases you, it will assigned in the order they’ve been found in the declaration to the shader output. Global variables can be inputs of a shader node. Global variables are forced to be vec4 and are defaulted to the value of the attribute inPosition in the vertex shader and vec4(1.0)(opaque white color) in the fragment shader.
  - The name of a previous shader node : this must be followed by and output variable of a the named shader node. This is what allows one to plug outputs from a node to inputs of another.
- VarName : the name of the variable to assign to the input. This variable must be known in name space declared before.
- MappingCondition : Follows the same rules as the activation condition for the shaderNode, this mapping will be embed in a #ifdef statement n the resulting shader.
OutputMapping : This block is optional, as mapping of output will be done in input mapping block of following shaderNodes, ecept if you want to output a value to the Global output of the shader.
- NameSpace : the name space of the output to assign, this can only be “Global” here.
- VarName : the name of a global output (can be anything, just be aware that 2 different names result in 2 different outputs).
- OutputVariable : Must be an output of the current node’s definition.
- MappingCondition : Same as before.

Complete material definition and Shader Nodes example

Here is an example of a very simple Material definition that just displays a solid color (controlled by a material parameter) on a mesh.

Shader Nodes only work if there is no shader declared in the technique. If you want to bypass the Shader Nodes, you can put a VertexShader and a FragmentShader statement in the technique and the shader nodes will be ignored.

MaterialDef Simple {
    MaterialParameters {
        Color Color
    }
    Technique {
        WorldParameters {
            WorldViewProjectionMatrix
        }
        VertexShaderNodes {
            ShaderNode CommonVert {
                Definition : CommonVert : Common/MatDefs/ShaderNodes/Common/CommonVert.j3sn
                InputMappings {
                    worldViewProjectionMatrix = WorldParam.WorldViewProjectionMatrix
                    modelPosition = Global.position.xyz
                }
                OutputMappings {
                    Global.position = projPosition
                }
            }
        }
        FragmentShaderNodes {
            ShaderNode ColorMult {
                Definition : ColorMult : Common/MatDefs/ShaderNodes/Basic/ColorMult.j3sn
                InputMappings {
                    color1 = MatParam.Color
                    color2 = Global.color
                }
                OutputMappings {
                    Global.color = outColor
                }
            }
        }
    }
}

This Material definition has one Default technique with 2 node declarations.
CommonVert Definition
CommonVert is a vertex shader node that has commonly used input and outputs of a vertex shader. It also computes the position of the vertex in projection space.
Here is the definition content (Common/MatDefs/ShaderNodes/Common/CommonVert.j3sn) :

ShaderNodesDefinitions {
    ShaderNodeDefinition CommonVert {
        Type: Vertex
        Shader GLSL100: Common/MatDefs/ShaderNodes/Common/commonVert.vert
        Documentation {
            This Node is responsible for computing vertex position in projection space.
            It also can pass texture coordinates 1 & 2, and vertexColor to the frgment shader as varying (or inputs for glsl >=1.3)
            @input modelPosition the vertex position in model space (usually assigned with Attr.inPosition or Global.position)
            @input worldViewProjectionMatrix the World View Projection Matrix transforms model space to projection space.
            @input texCoord1 The first texture coordinates of the vertex (usually assigned with Attr.inTexCoord)
            @input texCoord2 The second texture coordinates of the vertex (usually assigned with Attr.inTexCoord2)
            @input vertColor The color of the vertex (usually assigned with Attr.inColor)
            @output projPosition Position of the vertex in projection space.(usually assigned to Global.position)
            @output vec2 texCoord1 The first texture coordinates of the vertex (output as a varying)
            @output vec2 texCoord2 The second texture coordinates of the vertex (output as a varying)
            @output vec4 vertColor The color of the vertex (output as a varying)
        }
        Input{
            vec3 modelPosition
            mat4 worldViewProjectionMatrix
            vec2 texCoord1
            vec2 texCoord2
            vec4 vertColor
        }
        Output{
            vec4 projPosition
            vec2 texCoord1
            vec2 texCoord2
            vec4 vertColor
        }
    }
}

Note that texCoord1/2 and vertColor are declared both as input and output. The generator will use the same variables for them.

Here is the shader Node code ( Common/MatDefs/ShaderNodes/Common/commonVert.vert).

void main(){
    projPosition = worldViewProjectionMatrix * vec4(modelPosition, 1.0);
}

As you can see all the inputs and outputs are not used. That’s because most of them are attributes meant to be passed to the fragment shader as varyings. All the wiring will be handled by the generator only if those variables are used in an input or output mapping.

CommonVert input mapping
Here we have the most basic yet mandatory thing in a vertex shader, computing vertex position in projection space. For this we have 2 mappings :

worldViewProjectionMatrix = WorldParam.WorldViewProjectionMatrix : the input parameter worldViewProjectionMatrix is assigned with the WorldViewProjectionMatrix World parameter declared in the WorlParameters block of the technique.
modelPosition = Global.position.xyz : the modelPosition (understand the vertex position in the model coordinate space) is assigned with the Global position variable.

As mentioned before Global position is initialized with the attribute inPosition, so this is equivalent to :

modelPosition = Attr.inPosition.xyz

Also note the swizzle of the Global.position variable. ModelPosition is a vec3 and GlobalPosition is a vec4 so we just take the first 3 components.

CommonVert output mapping

Global.position = projPosition : The result of the multiplication of the worldViewProjectionMatrix and the modelPosition is assigned to the Globale position.

The Global.position variable will be assigned to the gl_Position glsl built in output at the end of the shader.

ColorMult Definition
ColorMult is a very basic Shader Node that takes two colors as input and multiply them. Here is the definition content (Common/MatDefs/ShaderNodes/Basic/ColorMult.j3sn) :

ShaderNodeDefinitions{
    ShaderNodeDefinition ColorMult {
        Type: Fragment
        Shader GLSL100: Common/MatDefs/ShaderNodes/Basic/colorMult.frag
        Documentation{
            Multiplies two colors
            @input color1 the first color
            @input color2 the second color
            @output outColor the resulting color
        }
        Input {
            vec4 color1
            vec4 color2
        }
        Output {
            vec4 outColor
        }
    }
}

Here is the shader Node code (Common/MatDefs/ShaderNodes/Basic/colorMult.frag).

void main(){
    outColor = color1 * color2;
}

ColorMult input mapping
All inputs are mapped here :

color1 = MatParam.Color : The first color is assigned to the Color Material parameter declared in the MaterialParameter block of the material definition.
color2 = Global.color : The second color is assigned with the Global color variable. this is defaulted to vec4(1.0) (opaque white).

In a very complex material def this variable could already have been assigned with a previous Shader Node output.

ColorMult output mapping

Global.color = outColor : the resulting color is assigned to the Global color variable.

Note that the Global.color variable will be assigned to gl_FragColor (glsl < 1.5) or declared as a Global ouput of the shader (glsl >= 1.5).

Also note that in case several Global variables are declared, the generator will assign them gl_FragData[i](glsl < 1.5) i being the order the variable has been found in the material def. For glsl >= 1.5 the veriable will just all be declared as shader output in the order they’ve been found in the declaration.

Generated shader code

Don’t take this code as carved in stone, the generated code can change as optimization of the shader generator goes on.

Vertex Shader (glsl 1.0)

uniform mat4 g_WorldViewProjectionMatrix;

attribute vec4 inPosition;

void main(){
        vec4 Global_position = inPosition;

        //CommonVert : Begin
        vec3 CommonVert_modelPosition = Global_position.xyz;
        vec4 CommonVert_projPosition;
        vec2 CommonVert_texCoord1;
        vec2 CommonVert_texCoord2;
        vec4 CommonVert_vertColor;

        CommonVert_projPosition = g_WorldViewProjectionMatrix * vec4(CommonVert_modelPosition, 1.0);
        Global_position = CommonVert_projPosition;
        //CommonVert : End

        gl_Position = Global_position;
}

All materials parameter, world parameters, attributes varying are declared first. then for each shader node, the declarative part is appended.
For the main function, for each shader node, the input mappings are declared and assigned, the output are declared.
Then the variable names are replaced in the sahder node code with there complete name (NameSpace_varName), material parameters are replaced in the shader code as is.
Then, the output are mapped.

As you can see texCoord1/2 and vertColor are declared but never used. That’s because the generator is not aware of that. By default it will declare all inputs in case they are used in the shaderNode code. Note that most glsl compiler will optimize this when compiling the shader on the GPU.

Fragment Shader (glsl 1.0)

uniform vec4 m_Color;

void main(){
        vec4 Global_color = vec4(1.0);

        //ColorMult : Begin
        vec4 ColorMult_color2 = Global_color;
        vec4 ColorMult_outColor;

        ColorMult_outColor = m_Color * ColorMult_color2;
        Global_color = ColorMult_outColor;
        //ColorMult : End

        gl_FragColor = Global_color;
}

Same as for the Vertex shader. Note that the color1 is not declared, because it’s directly replaced by the material parameter.

As a rule of thumb you should not assign a value to an input. Inputs are likely to be material paramters or outputs from other shaders and modifying them may cause unexpected behavior, even failure in your resulting shader.

For more explanations and design decisions please refer to the spec here https://docs.google.com/document/d/1S6xO3d1TBz0xcKe_MPTqY9V-QI59AKdg1OGy3U-HeVY/edit?usp=sharing.

Thank you for the brave ones that came through all this reading. I’m not gonna offer you a prize in exchange of a password, because we ran out of JME thongs…