Image and raw image components in Unity - Deep Dive

Overview - Image and raw image components

What is it?

In Unity, Image and Raw Image components are used to display pictures on the screen. The Image component shows sprites, which are images optimized for 2D games and UI. Raw Image displays textures directly, which can be any image data, like photos or videos. Both are part of Unity's UI system and help create interactive and visual elements.

Why it matters

Without Image and Raw Image components, it would be hard to show pictures or graphics in your game or app's interface. They let you add buttons, backgrounds, icons, and more, making the experience clear and engaging. Without them, UI would be plain and confusing, losing user interest and usability.

Where it fits

Before learning these, you should understand Unity basics like GameObjects and Components. After this, you can learn about UI layouts, animations, and scripting interactions to make dynamic interfaces.

Mental Model

Core Idea

Image shows optimized 2D sprites for UI, while Raw Image displays any texture directly, both rendering visuals on Unity's canvas.

Think of it like...

Think of Image as a framed photo printed and ready to hang, perfectly sized and styled, while Raw Image is like a raw photo print you can crop or edit before framing.

┌───────────────┐       ┌───────────────┐
│   Image       │       │  Raw Image    │
│ (Sprite UI)   │       │ (Texture UI)  │
│ - Uses Sprite │       │ - Uses Texture│
│ - Optimized   │       │ - Raw data    │
└──────┬────────┘       └──────┬────────┘
       │                       │
       │                       │
       ▼                       ▼
  Rendered on Canvas       Rendered on Canvas

Build-Up - 6 Steps

1

Foundation

Understanding Unity UI Canvas

Concept: Learn what the Canvas is and how UI elements are drawn on it.

The Canvas is a special GameObject in Unity that holds all UI elements like buttons, text, and images. It acts like a drawing board where UI components are placed and rendered. Without a Canvas, UI elements won't show on the screen.

Result

You know where UI elements live and how they appear in Unity scenes.

Understanding the Canvas is key because Image and Raw Image components only work inside it, anchoring your visuals to the screen.

2

Foundation

Basics of Image Component

Concept: Image component displays sprites, which are 2D images optimized for UI.

Add an Image component to a UI GameObject to show a sprite. Sprites are images imported with settings that optimize them for UI use, like transparency and slicing. You can change the sprite, color tint, and fill method to create effects.

Result

You can display and customize simple UI images like icons or buttons.

Knowing that Image uses sprites helps you prepare your graphics correctly for crisp and efficient UI rendering.

3

Intermediate

Exploring Raw Image Component

Concept: Raw Image shows textures directly, allowing more flexible image sources.

Raw Image takes a Texture (like PNG, JPG, or video frames) and displays it on UI. Unlike Image, it doesn't require sprites and can show any texture data. This is useful for dynamic images or video playback in UI.

Result

You can display photos, videos, or dynamically generated textures in your UI.

Understanding Raw Image lets you handle more complex or raw visual data beyond static sprites.

4

Intermediate

Comparing Image vs Raw Image Usage

🤔Before reading on: Do you think Image or Raw Image is better for animated UI icons? Commit to your answer.

Concept: Learn when to use Image or Raw Image based on your needs.

Image is best for UI sprites with built-in features like slicing and tinting. Raw Image is better for raw textures or video. For animated icons, Image supports sprite animation easily, while Raw Image requires manual texture swapping.

Result

You can choose the right component for your UI visuals effectively.

Knowing the strengths and limits of each component prevents performance issues and simplifies your UI design.

5

Advanced

Optimizing UI with Image Components

🤔Before reading on: Do you think using many Raw Images affects performance more than Images? Commit to your answer.

Concept: Learn how Image components optimize rendering and how to use them efficiently.

Image components use sprite atlases and batching to reduce draw calls, improving performance. Raw Images do not batch as well because textures can vary. Using sliced sprites in Image allows scalable UI without extra textures.

Result

Your UI runs smoother and uses less memory when optimized.

Understanding rendering optimization helps you build UI that performs well on all devices.

6

Expert

Advanced Texture Handling in Raw Image

🤔Before reading on: Can Raw Image display non-rectangular textures with transparency as easily as Image? Commit to your answer.

Concept: Explore how Raw Image handles textures, transparency, and shaders internally.

Raw Image displays textures as-is, including alpha transparency, but lacks built-in sprite features like slicing. You can apply custom shaders to Raw Image for effects like masking or color correction. Handling non-rectangular textures requires manual alpha management.

Result

You can create advanced UI effects using Raw Image with custom shaders and texture tricks.

Knowing Raw Image internals unlocks powerful customization beyond standard UI capabilities.

Under the Hood

Image components use sprites, which are textures with metadata for slicing and packing. Unity batches these sprites to reduce rendering overhead. Raw Image uses textures directly without sprite metadata, so it renders each texture separately. Both render on the Canvas using Unity's UI rendering pipeline, which converts UI elements into meshes drawn by the GPU.

Why designed this way?

Unity separated Image and Raw Image to optimize common UI needs (sprites) while allowing flexibility for raw textures. Sprites enable efficient batching and scaling, crucial for performance. Raw Image supports use cases like video or dynamic textures where sprite features are unnecessary or limiting.

Canvas
  │
  ├─ Image Component (Sprite)
  │     ├─ Uses Sprite Atlas
  │     ├─ Supports Slicing & Tint
  │     └─ Batched Rendering
  └─ Raw Image Component (Texture)
        ├─ Uses Raw Texture
        ├─ Supports Alpha Transparency
        └─ Separate Draw Calls

Myth Busters - 4 Common Misconceptions

Quick: Do you think Raw Image automatically supports sprite slicing like Image? Commit yes or no.

Common Belief: Raw Image supports all the same features as Image, including slicing and tinting.

Reality: Raw Image does not support sprite slicing or advanced sprite features; it only displays raw textures.

Why it matters: Using Raw Image expecting slicing causes UI to look broken or unscalable, leading to poor user experience.

Quick: Do you think using many Raw Images is as efficient as many Images? Commit yes or no.

Common Belief: Raw Image and Image have similar performance in UI rendering.

Reality: Image components batch draw calls efficiently; Raw Images usually cause more draw calls and lower performance.

Why it matters: Ignoring this can cause frame rate drops and lag in UI-heavy applications.

Quick: Can you use any texture as a sprite directly in Image? Commit yes or no.

Common Belief: Any texture can be assigned directly to an Image component without conversion.

Reality: Image requires textures to be imported as sprites; raw textures must be converted first.

Why it matters: Not converting textures leads to errors or missing images in UI.

Quick: Does Raw Image automatically handle video playback? Commit yes or no.

Common Belief: Raw Image can play videos by default.

Reality: Raw Image can display video textures but needs scripting and video player components to update textures.

Why it matters: Assuming automatic video playback causes confusion and wasted development time.

Expert Zone

1

Image components benefit from sprite atlases which pack many sprites into one texture, drastically reducing draw calls.

2

Raw Image can be combined with custom shaders to create UI effects like dynamic masking or color grading not possible with Image.

3

Using Image with sliced sprites allows UI elements to scale without distortion, a subtle but crucial detail for polished interfaces.

When NOT to use

Avoid Raw Image when you need efficient UI rendering or scalable sprites; use Image instead. Avoid Image when displaying dynamic textures like video or procedural textures; use Raw Image with scripting.

Production Patterns

In production, designers use Image for buttons, icons, and backgrounds with sprite atlases for performance. Raw Image is used for video players, webcam feeds, or dynamically generated textures. Combining both with animation and shaders creates rich, interactive UI.

Connections

Sprite Atlases

Image components rely on sprite atlases for efficient rendering.

Understanding sprite atlases helps optimize UI performance by reducing draw calls when using Image components.

Shader Programming

Raw Image can use custom shaders to modify texture appearance dynamically.

Knowing shaders allows advanced UI effects on Raw Images, expanding creative possibilities.

Video Streaming Technology

Raw Image displays video textures updated by video streaming components.

Understanding video streaming helps integrate live or recorded video into UI using Raw Image.

Common Pitfalls

#1 Assigning a raw texture directly to an Image component without converting to sprite.

Wrong approach:

imageComponent.sprite = myTexture;

Correct approach:

imageComponent.sprite = Sprite.Create(myTexture, new Rect(0,0,myTexture.width,myTexture.height), new Vector2(0.5f,0.5f));

Root cause:

Image component requires a Sprite type, not a Texture, so direct assignment fails.

#2 Using many Raw Image components for UI icons causing performance drops.

Wrong approach:

Create 100 Raw Images each with separate textures for UI icons.

Correct approach:

Use Image components with sprites packed in a sprite atlas for icons.

Root cause:

Raw Images do not batch well, increasing draw calls and reducing performance.

#3 Expecting Raw Image to automatically handle video playback without scripting.

Wrong approach:

Assign video clip texture to Raw Image and expect playback.

Correct approach:

Use VideoPlayer component to update Raw Image texture each frame.

Root cause:

Raw Image only displays textures; video playback requires updating textures via scripting.

Key Takeaways

Image and Raw Image components display visuals on Unity's UI Canvas but serve different purposes.

Image uses sprites optimized for UI with features like slicing and batching for performance.

Raw Image displays raw textures directly, useful for videos or dynamic images but less optimized.

Choosing the right component affects UI quality, performance, and flexibility.

Understanding their differences and internals empowers you to build efficient and rich user interfaces.