Tech

Moving from OpenGL to Metal

Moving from OpenGL to Metal

Metal was launched in 2014 as a common function API for GPU-based computation. In 2018, Apple deprecated OpenGL in iOS 12 for each iOS and macOS.

On this tutorial, you’ll find out how to convert an app from utilizing OpenGL to Metal. To finish this tutorial, you’ll want a working OpenGL app. The starter challenge for this tutorial is the completed undertaking from the OpenGL tutorial. You possibly can obtain what you want utilizing the Obtain Supplies button discovered on the prime or backside of this tutorial.

Earlier than getting began, it’s your decision to take a look at these nice assets on Metal and OpenGL.

Should you don’t have expertise with 3D graphics, don’t fear! You’ll nonetheless give you the option to comply with alongside. Should you do have some expertise with 3D programming or OpenGL, you could discover this tutorial simpler. Most of the similar ideas apply in Metal.

Word: Metal apps don’t run on the iOS simulator. They require a tool with an Apple A7 chip or later. To finish this tutorial, you’ll want an A7 gadget or newer.

OpenGL ES vs. Metal

OpenGL ES is designed to be a cross-platform framework. Meaning, with a number of small modifications, you possibly can run C++ OpenGL ES code on different platforms, similar to Android.

The cross-platform help of OpenGL ES is good, however Apple realized it was lacking the signature integration of the working system, hardware, and software program that each one good Apple merchandise have. So, it took a clean-room strategy and designed a graphics API particularly for Apple hardware. The aim was to have low overhead and excessive efficiency whereas supporting the newest and biggest options.

The result’s Metal, which may present up to 10✕ the variety of draw calls in your app in contrast to OpenGL ES. The consequences are superb — you could keep in mind them from the Zen Backyard instance within the WWDC 2014 keynote.

Understanding Conceptual Variations

From a improvement perspective, OpenGL and Metal are comparable. In each, you arrange buffers with knowledge to move to the GPU and specify vertex and fragment shaders. In OpenGL tasks, there’s a GLKBaseEffect, which is an abstraction on prime of shaders. There’s no such API for Metal, so that you want to write shaders your self. However don’t fear – it’s not too difficult!

The most important distinction between OpenGL and Metal is that in Metal, you often function with two kinds of objects:

  1. Descriptor objects.
  2. Compiled-state objects.

The thought is straightforward. You create a descriptor object and compile it. The compiled-state object is a GPU-optimized useful resource. The creation and compilation are each costly operations, so the thought is to do them as not often as attainable, and later to function with compiled-state objects.

This strategy signifies that when utilizing Metal, you don’t want to do lots of setup operations on the render loop. This makes it far more environment friendly than OpenGL which may’t do the identical due to structure restrictions.

Time to discover the variations by yourself!

Getting Began

As a reminder, obtain the information you want utilizing the Obtain Supplies button on the prime or backside of this tutorial. Then, open OpenGLKit.xcodeproj within the Starter folder. You’ll see a venture which makes use of OpenGL.

Construct & run.

Metal Square

You must see a colourful sq. spinning. This sq. is rendered with OpenGL. Nevertheless, because the deployment goal for this challenge is iOS 12, there are a number of OpenGL deprecation warnings. You possibly can see these in Xcode within the Situation navigator.

Now you’re going to draw the identical sq. however with Metal. And eliminate all these pesky warnings!

Integrating Metal

Open ViewController.swift, and alter ViewController to be a subclass of UIViewController as an alternative of GLKViewController. In Metal, there’s no such factor as MetalViewController. As an alternative, you might have to use MTKView contained in the UIViewController.

MTKView is part of the MetalKit framework. To entry this API,
add the next on the prime of the file:


import MetalKit

Switching From OpenGL

Now it’s time to do some OpenGL cleanup. Comply with these steps:

1. Rename each occurrences of setupGL() to setupMetal().
2. Take away tearDownGL() and deinit() strategies. With Metal, there’s no want for specific cleanup like this.
three. Discover and take away the entire extension GLKViewControllerDelegate, since this view controller is not a GLKViewController. Word that glkViewControllerUpdate incorporates the logic for spinning. That is helpful, however for now, take away it.
four. Take away the next code from the highest of setupMetal():


context = EAGLContext(api: .openGLES3)
EAGLContext.setCurrent(context)

if let view = self.view as? GLKView, let context = context
view.context = context
delegate = self

5. Take away the next properties from the highest of ViewController:


personal var context: EAGLContext?
personal var impact = GLKBaseEffect()
personal var ebo = GLuint()
personal var vbo = GLuint()
personal var vao = GLuint()

Lastly, on the prime of the ViewController class declaration, add an outlet to a MTKView:


@IBOutlet weak var metalView: MTKView!

Setting Up the Storyboard

The ViewController is not GLKViewController, so that you want to make some modifications within the storyboard.

Open Most important.storyboard. On this instance, the storyboard incorporates two scenes, each named View Controller Scene. One has a GLKView, and the opposite one accommodates a MTKView and a connection to the outlet that you simply’ve simply added to the supply code.

All you want to do is about the scene with the MTKView because the preliminary View Controller. Discover the scene which doesn’t at present have the arrow pointing to it. Click on on the bar on the prime to choose the view controller. Alternatively you possibly can choose it within the doc define pane. Then open the attributes inspector and verify Is Preliminary View Controller.

As soon as that’s completed, you’ll be able to delete the primary scene. Good work!

Setting Up Metal

Are you prepared? It’s time to use some Metal!

In Metal, the primary object that you simply’ll use to entry the GPU is MTLDevice. The subsequent most necessary object is MTLCommandQueue. This object is a queue to which you’ll move encoded frames.

Open ViewController.swift and add these properties:


personal var metalDevice: MTLDevice!
personal var metalCommandQueue: MTLCommandQueue!

Now, go to setupMetal(). Exchange the contents of it with the next:


metalDevice = MTLCreateSystemDefaultDevice()
metalCommandQueue = metalDevice.makeCommandQueue()
metalView.system = metalDevice
metalView.delegate = self

That’s so much shorter than what was there earlier than proper!

This grabs the system default Metal system, then makes a command queue from the gadget. Then it assigns the gadget to the Metal view. Lastly it units the view controller because the view’s delegate to obtain callbacks when to draw and resize.

Now you want to to implement the MTKViewDelegate protocol.

On the backside of ViewController.swift, add this extension:


extension ViewController: MTKViewDelegate
// 1
func mtkView(_ view: MTKView, drawableSizeWillChange measurement: CGSize)


// 2
func draw(in view: MTKView)

This extension implements two strategies.

  1. This technique is known as when the drawable measurement modifications, reminiscent of when the display rotates.
  2. This technique known as to carry out the precise drawing.

Primary Drawing

You’re all set to draw! To maintain issues easy, you’ll draw a grey background first.

For every body you need to attract Metal, you will need to create a command buffer through which you specify what and the way you need to draw. Then, this buffer is encoded on the CPU and despatched to the GPU by means of the command queue.

Add the next code inside draw(in:):


// 1
guard let drawable = view.currentDrawable else
return


let renderPassDescriptor = MTLRenderPassDescriptor() // 2
renderPassDescriptor.colorAttachments[0].texture = drawable.texture // three
renderPassDescriptor.colorAttachments[0].loadAction = .clear // four
renderPassDescriptor.colorAttachments[0]
.clearColor = MTLClearColor(purple: zero.85, inexperienced: zero.85, blue: zero.85, alpha: 1.zero) // 5

// 6
guard let commandBuffer = metalCommandQueue.makeCommandBuffer() else
return


// 7
guard let renderEncoder = commandBuffer
.makeRenderCommandEncoder(descriptor: renderPassDescriptor) else
return


// Body drawing goes right here

renderEncoder.endEncoding() // Eight

commandBuffer.current(drawable) // 9
commandBuffer.commit() // 10

This can be a massive one. Right here’s what’s going on within the code above:

  1. Guarantee there’s a legitimate drawable to be used for the present body.
  2. MTLRenderPassDescriptor incorporates a set of attachments which are the rendering vacation spot for pixels generated by a rendering move.
  3. Set the feel from the view as a vacation spot for drawing.
  4. Clear each pixel at the beginning of a rendering.
  5. Specify the colour to use when the colour attachment is cleared. On this case, it’s a stunning 85% grey.
  6. Ask the command queue to create a brand new command buffer.
  7. Create an encoder object that may encode graphics rendering instructions into the command buffer. You’ll add your precise drawing code after this assertion later.
  8. Declare that each one command generations from this encoder are full.
  9. Register a drawable presentation to happen as quickly as potential.
  10. Commit this command buffer for execution within the command queue.

In abstract, you create a command buffer and a command encoder. Then, you do your drawing within the command encoder and commit the command buffer to the GPU by way of the command queue. These steps are then repeated every time the body is drawn.

Observe: When you’re utilizing the simulator, at this level you’ll get a construct error. You want a suitable system to construct and run the app.

Construct and run the app.

Metal Gray View

What a stunning gray shade!

Drawing Primitives

It should take a while to draw one thing extra significant. So, take a deep breath and dive proper in!

To attract one thing, you should cross knowledge that represents the thing to the GPU. You have already got Vertex construction to characterize the vertices knowledge, however you want to make a small change to use it with Metal.

Open ViewController.swift and alter the Indices property from this:


var Indices: [GLubyte]

To this:


var Indices: [UInt32]

You’ll see why this modification is required when drawing primitives.

Knowledge Buffers

To move vertices knowledge to the GPU, you want to create two buffers: one for vertices and one for indices. That is comparable to OpenGL’s component buffer object (EBO) and vertex buffer object (VBO).

Add these properties to ViewController:


personal var vertexBuffer: MTLBuffer!
personal var indicesBuffer: MTLBuffer!

Now, inside setupMetal(), add the next on the backside:


let vertexBufferSize = Vertices.measurement()
vertexBuffer = metalDevice
.makeBuffer(bytes: &Vertices, size: vertexBufferSize, choices: .storageModeShared)

let indicesBufferSize = Indices.measurement()
indicesBuffer = metalDevice
.makeBuffer(bytes: &Indices, size: indicesBufferSize, choices: .storageModeShared)

This asks metalDevice to create the vertices and indices buffers initialized together with your knowledge.

Now, go to draw(in:), and proper earlier than:


renderEncoder.endEncoding()

Add the next:


renderEncoder.setVertexBuffer(vertexBuffer, offset: zero, index: zero)
renderEncoder.drawIndexedPrimitives(
sort: .triangle,
indexCount: Indices.rely,
indexType: .uint32,
indexBuffer: indicesBuffer,
indexBufferOffset: zero)

First this passes the vertex buffer to the GPU, setting it at index zero. Then, it attracts triangles utilizing indicesBuffer. Notice that you simply want to specify the index sort which is uint32. That’s why you modified the indices sort earlier.

Construct and run the app.

Crash! That’s not good. You handed knowledge from the CPU to the GPU. It crashed since you didn’t specify how the GPU ought to use this knowledge. You’ll want to add some shaders! Fortuitously, that’s the subsequent step.

Including Shaders

Create a brand new file. Click on File ▸ New ▸ File…, select iOS ▸ Supply ▸ Metal File. Click on Subsequent. Identify it Shaders.metallic and reserve it wherever you need.

Metal makes use of C++ to write shaders. Normally, it’s comparable to the GLSL used for OpenGL.

For extra on shaders, verify the references on the backside of this tutorial.

Add this to the underside of the file:


struct VertexIn
packed_float3 place;
packed_float4 colour;
;

struct VertexOut
float4 computedPosition [[position]];
float4 shade;
;

You’ll use these buildings as enter and output knowledge for the vertex shader.

Writing a Vertex Shader

A vertex shader is a perform that runs for every vertex that you simply draw.

Under the structs, add the next code:


vertex VertexOut basic_vertex( // 1
const system VertexIn* vertex_array [[ buffer(0) ]], // 2
unsigned int vid [[ vertex_id ]]) // three
// four
VertexIn v = vertex_array[vid];

// 5
VertexOut outVertex = VertexOut();
outVertex.computedPosition = float4(v.place, 1.zero);
outVertex.colour = v.colour;
return outVertex;

  1. The vertex signifies this can be a vertex shader perform. The return sort for this shader is VertexOut.
  2. Right here, you get the vertex buffer that you simply handed to the command encoder.
  3. This parameter is a vertex id for which this shader was referred to as.
  4. Seize the enter vertex for the present vertex id.
  5. Right here, you create a VertexOut and move knowledge from the present VertexIn. That is merely utilizing the identical place and colour because the enter.

At this level, the vertex shader’s job is completed.

Writing a Fragment Shader

After the vertex shader finishes, the fragment shader runs for every potential pixel.

Under the vertex shader, add the next code:


fragment float4 basic_fragment(VertexOut interpolated [[stage_in]])
return float4(interpolated.colour);

The fragment shader receives the output from the vertex shader — the VertexOut. Then, the fragment shader returns the colour for the present fragment.

Hooking up the Shaders to the Pipeline

The shaders are in place, however you haven’t hooked them to your pipeline but. So as to do this, return to ViewController.swift, and add this property to the category:


personal var pipelineState: MTLRenderPipelineState!

This property will include shaders knowledge.

Now, discover setupMetal(). Add the next on the backside of the tactic:


// 1
let defaultLibrary = metalDevice.makeDefaultLibrary()!
let fragmentProgram = defaultLibrary.makeFunction(identify: “basic_fragment”)
let vertexProgram = defaultLibrary.makeFunction(identify: “basic_vertex”)

// 2
let pipelineStateDescriptor = MTLRenderPipelineDescriptor()
pipelineStateDescriptor.vertexFunction = vertexProgram
pipelineStateDescriptor.fragmentFunction = fragmentProgram
pipelineStateDescriptor.colorAttachments[0].pixelFormat = .bgra8Unorm

// three
pipelineState = attempt! metalDevice
.makeRenderPipelineState(descriptor: pipelineStateDescriptor)

That is what the code does:

  1. Finds the vertex and fragment shaders by their identify in all .metallic information.
  2. Creates a descriptor object with the vertex shader and the fragment shader. The pixel format is about to a regular BGRA (Blue Inexperienced Purple Alpha) Eight-bit unsigned.
  3. Asks the GPU to compile all that into the GPU-optimized object.

Now, the pipeline state is prepared. It’s time to use it. For this, go to draw(in:), and proper earlier than:


renderEncoder.drawIndexedPrimitives(
sort: .triangle,
indexCount: Indices.rely,
indexType: .uint32,
indexBuffer: indicesBuffer,
indexBufferOffset: zero)

Add:


renderEncoder.setRenderPipelineState(pipelineState)

Construct and run. You must get this colourful display.

Metal Gradient Background

Matrices

So as to manipulate the scene, you want to cross the projection and model-view matrices to the GPU. The projection matrix permits you to manipulate the notion of the scene to make nearer objects seem greater than farther ones. The mannequin view matrix permits you to manipulate the place, rotation, and scale of an object or the entire scene.

So as to use these matrices, you’ll create a brand new struct. Open Vertex.swift. On the prime of the file, add:


struct SceneMatrices
var projectionMatrix: GLKMatrix4 = GLKMatrix4Identity
var modelviewMatrix: GLKMatrix4 = GLKMatrix4Identity

Observe that you simply’ll nonetheless use GLKMatrix4. This a part of GLKit just isn’t deprecated, so you should use it for matrices in Metal.

Now, open ViewController.swift, and add two new properties:


personal var sceneMatrices = SceneMatrices()
personal var uniformBuffer: MTLBuffer!

Then, go to draw(in:), and proper earlier than:


renderEncoder.setRenderPipelineState(pipelineState)

Add:


// 1
let modelViewMatrix = GLKMatrix4MakeTranslation(zero.zero, zero.zero, -6.zero)
sceneMatrices.modelviewMatrix = modelViewMatrix

// 2
let uniformBufferSize = MemoryLayout.measurement(ofValue: sceneMatrices)
uniformBuffer = metalDevice.makeBuffer(
bytes: &sceneMatrices,
size: uniformBufferSize,
choices: .storageModeShared)

// three
renderEncoder.setVertexBuffer(uniformBuffer, offset: zero, index: 1)

Right here’s what the code above does:

  1. Creates a matrix to shift the thing backwards by 6 models, to make it look smaller.
  2. Creates a uniform buffer such as you did with vertex buffer earlier than, however with matrices knowledge.
  3. Hooks the uniform buffer to the pipeline and units its identifier to 1.

Projection Matrix

When you’re nonetheless in ViewController.swift, inside mtkView(_:drawableSizeWillChange:), add the next:


let facet = fabsf(Float(measurement.width) / Float(measurement.peak))
let projectionMatrix = GLKMatrix4MakePerspective(
GLKMathDegreesToRadians(65.zero),
facet,
four.zero,
10.zero)
sceneMatrices.projectionMatrix = projectionMatrix

This code creates a projection matrix based mostly on the facet ratio of the view. Then, it assigns it to the scene’s projection matrix.

With this in place, your sq. will now look sq. and never stretched out to the entire display. :]

Matrices in Shaders

You’re virtually there! Subsequent, you’ll want to obtain matrices knowledge in shaders. Open Shaders.metallic. On the very prime, add a brand new struct:


struct SceneMatrices
float4x4 projectionMatrix;
float4x4 viewModelMatrix;
;

Now, substitute the basic_vertex perform with the next:


vertex VertexOut basic_vertex(
const system VertexIn* vertex_array [[ buffer(0) ]],
const gadget SceneMatrices& scene_matrices [[ buffer(1) ]], // 1
unsigned int vid [[ vertex_id ]])
// 2
float4x4 viewModelMatrix = scene_matrices.viewModelMatrix;
float4x4 projectionMatrix = scene_matrices.projectionMatrix;

VertexIn v = vertex_array[vid];

// three
VertexOut outVertex = VertexOut();
outVertex
.computedPosition = projectionMatrix * viewModelMatrix * float4(v.place, 1.zero);
outVertex.shade = v.shade;
return outVertex;

Right here’s what has modified:

  1. Receives matrices as a parameter contained in the vertex shader.
  2. Extracts the view mannequin and projection matrices.
  3. Multiplies the place by the projection and consider mannequin matrices.

Construct and run the app. You must see this:

Gradient Static Cube

W00t! A sq.!

Making it Spin

Within the OpenGL implementation, GLViewController offered lastUpdateDate which might inform you when the final render was carried out. In Metal, you’ll have to create this your self.

First, in ViewController, add a brand new property:


personal var lastUpdateDate = Date()

Then, go to draw(in: ), and simply earlier than:


commandBuffer.current(drawable)

Add the next code:


commandBuffer.addCompletedHandler _ in
self.lastUpdateDate = Date()

With this in place, when a body drawing completes, it updates lastUpdateDate to the present date and time.

Now, it’s time to spin! In draw(in:), substitute:


let modelViewMatrix = GLKMatrix4Translate(GLKMatrix4Identity, zero, zero, -6.zero)

With:


var modelViewMatrix = GLKMatrix4MakeTranslation(zero.zero, zero.zero, -6.zero)
let timeSinceLastUpdate = lastUpdateDate.timeIntervalSince(Date())

// 1
rotation += 90 * Float(timeSinceLastUpdate)

// 2
modelViewMatrix = GLKMatrix4Rotate(
modelViewMatrix,
GLKMathDegreesToRadians(rotation), zero, zero, 1)

This increments the rotation property by an quantity proportional to the time between the final render and this render. Then, it applies a rotation across the Z-axis to the model-view matrix.

Construct and run the app. You’ll see the dice spinning. Success!

Metal Spinning Cube

The place to Go From Right here?

Obtain the ultimate undertaking for this tutorial utilizing the Obtain Supplies button on the prime or backside of the tutorial.

Congrats! You’ve discovered a ton concerning the Metal API! Now you perceive a number of the most necessary ideas in Metal, resembling shaders, units, command buffers, and pipelines, and you’ve got some helpful insights into the variations between OpenGL and Metal.

For extra, make sure to take a look at these nice assets from Apple:

You additionally may take pleasure in this Starting Metal course, which covers these ideas intimately with video tutorials.

If studying is extra your type, take a look at the Metal Getting Began tutorial. For a whole ebook on the topic, see Metal by Tutorials.

I hope you’ve loved this tutorial! When you have any feedback or questions, please be a part of the discussion board dialogue under!