Custom shader fundamentals in Unity - Deep Dive

Overview - Custom shader fundamentals

What is it?

Custom shaders are small programs that tell your computer how to draw objects with special colors, lights, and effects in a game or app. They control how surfaces look by calculating colors and brightness for each pixel on the screen. Writing custom shaders lets you create unique visual styles beyond the default looks. They work inside Unity to make your scenes more vivid and interesting.

Why it matters

Without custom shaders, all objects would look plain and similar, limiting creativity and realism in games or apps. Custom shaders solve the problem of making materials look shiny, rough, glowing, or transparent in ways that default tools can't. They let artists and developers bring their vision to life with special effects that catch the eye and improve immersion. Without them, visuals would be dull and less engaging.

Where it fits

Before learning custom shaders, you should understand basic Unity concepts like materials, textures, and lighting. Knowing some graphics basics helps too. After mastering custom shaders, you can explore advanced topics like shader graph, compute shaders, and optimization techniques for better performance.

Mental Model

Core Idea

A custom shader is a tiny program that decides the color and brightness of every pixel on an object to create special visual effects.

Think of it like...

Imagine painting a wall with a magic brush that changes color and shine depending on how light hits it and where you look from.

┌───────────────┐
│   Vertex Data  │
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Vertex Shader │  (moves and prepares points)
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Fragment Shader│  (colors each pixel)
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Rendered Image │

Build-Up - 7 Steps

1

Foundation

What is a Shader in Unity

Concept: Introduces the basic idea of shaders as programs that control object appearance.

In Unity, a shader is a script that tells the graphics card how to draw an object. It controls colors, textures, and lighting effects. Unity uses shaders to make objects look realistic or stylized. The simplest shader just paints an object with a flat color.

Result

You understand that shaders are behind every visual effect on 3D objects in Unity.

Understanding that shaders are programs controlling visuals helps you see why customizing them changes how objects look.

2

Foundation

Shader Structure Basics

Concept: Explains the two main parts of a shader: vertex and fragment shaders.

A shader has two main parts: the vertex shader and the fragment shader. The vertex shader moves and prepares the points (vertices) of a 3D model. The fragment shader colors each pixel (fragment) on the surface. Together, they decide the final look of the object.

Result

You can identify the roles of vertex and fragment shaders in rendering.

Knowing these parts clarifies how 3D shapes become colored images on screen.

3

Intermediate

Writing a Simple Custom Shader

🤔Before reading on: do you think a shader needs to handle lighting calculations to show color? Commit to your answer.

Concept: Shows how to write a minimal shader that colors an object without lighting.

A simple shader can just assign a fixed color to every pixel. For example, in Unity's ShaderLab language, you write a fragment shader that returns a color like red. This shader ignores lighting and textures, so the object appears flat but colored.

Result

You create a shader that paints an object solid red regardless of light.

Understanding that shaders can be as simple as fixed colors helps you build up complexity step-by-step.

4

Intermediate

Using Textures in Shaders

🤔Before reading on: do you think textures are applied automatically or must be sampled in the shader? Commit to your answer.

Concept: Introduces how shaders use textures by sampling colors from images.

Textures are images wrapped on 3D objects. In a shader, you sample a texture using UV coordinates to get color data. The fragment shader reads the texture color for each pixel and uses it to color the object. This lets you add detailed patterns or photos to surfaces.

Result

Your shader can display images on objects, making them look more detailed.

Knowing that shaders must explicitly sample textures explains why texture mapping is flexible and powerful.

5

Intermediate

Adding Lighting Calculations

🤔Before reading on: do you think lighting is calculated per vertex or per pixel in shaders? Commit to your answer.

Concept: Explains how to add simple lighting to shaders for realistic shading.

Lighting makes objects look 3D by simulating how light hits surfaces. You can calculate lighting in the vertex shader (faster but less accurate) or fragment shader (slower but better). A common method is Lambertian diffuse lighting, which uses the angle between light and surface to adjust brightness.

Result

Your shader shows shading that changes with light direction, adding depth.

Understanding lighting math inside shaders is key to making objects look real.

6

Advanced

Shader Properties and Material Integration

🤔Before reading on: do you think shader variables can be changed from Unity's editor? Commit to your answer.

Concept: Shows how to expose variables in shaders so artists can tweak them in Unity materials.

Shaders can declare properties like colors, textures, or numbers. These appear in Unity's material inspector, letting you change shader behavior without editing code. For example, you can add a color property to control tint or a float to adjust brightness dynamically.

Result

You create flexible shaders that artists can customize easily in Unity.

Knowing how to connect shader code with Unity's editor improves collaboration and workflow.

7

Expert

Performance and Precision Considerations

🤔Before reading on: do you think using complex math in fragment shaders always improves visuals without cost? Commit to your answer.

Concept: Discusses trade-offs in shader complexity, precision, and performance in real projects.

Shaders run on the GPU and affect frame rate. Complex calculations in fragment shaders can slow rendering. Choosing between vertex and fragment calculations, using lower precision types, and minimizing texture lookups help optimize performance. Also, understanding GPU pipeline stages helps write efficient shaders.

Result

You can write shaders that balance visual quality and game performance.

Knowing GPU limits and optimization techniques prevents slowdowns and bugs in production.

Under the Hood

When a frame is rendered, Unity sends vertex data to the GPU. The vertex shader runs first, transforming each vertex's position and passing data like UVs and normals. Then the rasterizer breaks the shape into pixels. For each pixel, the fragment shader runs, calculating the final color using inputs like textures, lighting, and properties. The GPU executes these shaders massively in parallel for speed.

Why designed this way?

Shaders separate vertex and fragment stages to optimize performance and flexibility. Vertex shaders handle geometry transformations efficiently, while fragment shaders focus on pixel-level detail. This division matches the GPU's pipeline design, allowing parallel processing and hardware acceleration. Early graphics APIs were fixed-function, but programmable shaders gave developers creative control.

┌───────────────┐
│   CPU sends   │
│ vertex data   │
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Vertex Shader │  (transforms vertices)
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Rasterizer    │  (creates pixels)
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Fragment Shader│  (colors pixels)
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Framebuffer   │  (final image)
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Do you think shaders run on the CPU or GPU? Commit to your answer.

Common Belief: Shaders run on the CPU like normal code.

Reality: Shaders run on the GPU, which is specialized hardware for parallel graphics processing.

Why it matters: Thinking shaders run on the CPU leads to misunderstanding performance and debugging issues.

Quick: Do you think all shaders automatically handle lighting? Commit to your answer.

Common Belief: Shaders always include lighting calculations by default.

Reality: Shaders only do what you program; if you don't add lighting math, the object looks flat.

Why it matters: Assuming lighting is automatic can cause confusion when objects appear unlit or wrong.

Quick: Do you think textures are applied without code in shaders? Commit to your answer.

Common Belief: Textures are automatically mapped onto objects without shader code.

Reality: Shaders must explicitly sample textures using UV coordinates to show them.

Why it matters: Not knowing this causes errors where textures don't appear or look wrong.

Quick: Do you think increasing shader complexity always improves visuals? Commit to your answer.

Common Belief: More complex shaders always make graphics look better.

Reality: Complex shaders can hurt performance and sometimes cause visual glitches if not carefully managed.

Why it matters: Ignoring performance costs can make games lag or crash on weaker devices.

Expert Zone

1

Shader precision types (half, float, fixed) affect performance and quality subtly but critically in mobile vs desktop.

2

The order of operations in shaders can cause unexpected color blending or lighting artifacts if misunderstood.

3

Using built-in Unity shader variables and macros correctly can save time and avoid bugs but is often overlooked.

When NOT to use

Custom shaders are not ideal when you need rapid prototyping or simple visuals; Unity's Shader Graph or built-in shaders are better. Also, for extremely complex effects, compute shaders or post-processing may be more suitable.

Production Patterns

In production, shaders are often modularized with reusable functions, use property blocks for dynamic changes, and are optimized by minimizing texture lookups and branching. Artists and programmers collaborate by exposing shader properties for easy tweaking.

Connections

Computer Graphics Pipeline

Custom shaders are a programmable part of the graphics pipeline stages.

Understanding the full graphics pipeline helps you know where shaders fit and how data flows through rendering.

Functional Programming

Shaders are small pure functions that transform inputs to outputs without side effects.

Seeing shaders as pure functions clarifies why they must be deterministic and efficient.

Photography Lighting Techniques

Shader lighting calculations mimic real-world light behavior used in photography.

Knowing photography lighting helps you design shaders that create believable light and shadow effects.

Common Pitfalls

#1 Ignoring coordinate spaces causing lighting errors

Wrong approach:

float3 lightDir = normalize(_WorldSpaceLightPos0.xyz - IN.vertex);

Correct approach:

float3 lightDir = normalize(_WorldSpaceLightPos0.xyz - IN.worldPos);

Root cause:

Confusing vertex position space with world space leads to wrong lighting direction calculations.

#2 Sampling textures without UV coordinates

Wrong approach:

fixed4 col = tex2D(_MainTex, float2(0,0));

Correct approach:

fixed4 col = tex2D(_MainTex, IN.uv);

Root cause:

Using fixed UV coordinates ignores the actual texture mapping, causing uniform or wrong colors.

#3 Using high precision everywhere causing slow performance

Wrong approach:

float4 color = tex2D(_MainTex, IN.uv); // using float precision everywhere

Correct approach:

fixed4 color = tex2D(_MainTex, IN.uv); // use lower precision where possible

Root cause:

Not understanding precision types leads to unnecessary GPU load and battery drain on mobiles.

Key Takeaways

Custom shaders are small programs that control how every pixel on an object looks by calculating colors and lighting.

Shaders have two main parts: vertex shaders that handle geometry and fragment shaders that handle pixel colors.

You must write shader code to sample textures and calculate lighting; these are not automatic.

Balancing shader complexity and performance is crucial for smooth, beautiful graphics in games.

Exposing shader properties to Unity materials allows artists to customize visuals without changing code.