Setup a 3D Chessboard With SwiftUI and RealityKit | by Mark Lucking | Dec, 2022

Using SwiftUI, RealityKit, ARKit and MultiPeer Framework

I’ve spent the better part of this year [2022] Scenekit quest. I’ve documented a journey along the way with nearly two dozen articles on the subject that you’ll find here on Medium. After covering most of the elements of SceneKit, I decided to move on to RealityKit/ARKit in 2023.

Although I wasn’t sure where to start, I looked at the latest presentations on ARKit at WWDC2022 and then RealityKit and well – to no avail. I saw above that came first, followed by WWDC soon after.

I saw it, “Creating Apps with RealityKit”, from WWDC2019. It was a eureka moment, well, almost. I couldn’t help but consider the similarities between SceneKit and RealityKit. RealityKit is SceneKit. They may have developed it from the ground up with multiprocessing in mind, but the primitives, in many respects, look almost identical. They wanted to make the transition from one to the other as smooth as possible. And in fact, it looks like you can make a RealityKit-based app even without the camera if you do — an app that ends up looking like you did it on SceneKit. ARKit appears to be an integral part of RealityKit, so they’re like yin and yang; One is understandable only with the other.

After watching the presentation, I decided to recreate the chess game I described this paper Focusing on the first two sections using RealityKit/ARKit. Building a prototype and linking it to the game situation.


Bon – so it’s almost 2023, and there’s no point turning back the clock and using UIKit for me; The build plan will be the same as described in that WWDC2019 presentation – with some minor changes and updates.
I copied the code behind from my previous SceneKit article and used it as a basis to build on. My UIRepresentable was as simple as you can imagine, just ten lines or so.

struct CustomARView: UIViewRepresentable 
typealias UIViewType = ARView

var view:ARView
var options: [Any] = []

func makeUIView(context: Context) -> ARView
view.session.delegate = context.coordinator
return view

func updateUIView(_ view: ARView, context: Context)

func makeCoordinator() -> Coordinator

// coordinator

coordinator class

Equally concise with the coordinator class, here’s the skeleton code, nothing really to do with it.

class Coordinator: NSObject, ARSessionDelegate {
private let arView: ARView

init(_ view: ARView)

self.arView = view

At this point, it was a no brainer; I ran WWDC2019 on a screen and updated the code within the co-ordinator. I was building a prototype here, so, no need for the whole board, just needed to get the base code to understand the semantics and syntax.

I initially hit a minor snag with the scaling as my model monsters appeared, an easy fix. I hit a second snag with the syntax of transform which was different from what I expected, however, in a pleasant surprise, Apple has well documented here itAnd it was also resolved quickly.

Those were the basics – in fact, I went a step further because I already used 3D models for my prototype, only I used the wrong call to load them. I was doing it synchronously which caused my app to freeze. I reimplemented the async code with some help from this SO post, Here the variable loading is of cancelable type. [The WWDC2019 code won’t even compile],

loading = Entity.loadModelAsync(named: assetName)
.sink(receiveCompletion: completion in
if case let .failure(error) = completion
print("Unable to load a model due to error \(error)")


, receiveValue: [self] (entity: Entity) in
if let entity = entity as? ModelEntity
let piecePlayer = entity
print("Congrats! Model is successfully loaded!")
piecePlayer.position = SIMD3(x: 0, y: 0, z: 0)
piecePlayer.setScale(SIMD3(0.01,0.01,0.01), relativeTo: piecePlayer)
piecePlayer.generateCollisionShapes(recursive: true) = "GCHKing"


I still needed the board; In SceneKit, I made one out of cubes, but it made less sense with RealityKit, so I decided to use a plain mesh with this code. The code I have embedded in some loops and some booleans to get the complete board.

let material = SimpleMaterial(color: .black, isMetallic: true)
let plainMesh = MeshResource.generatePlane(width: 0.1, depth: 0.1)
let entity = ModelEntity(mesh: plainMesh, materials: [material])

I was halfway through WWDC2019 on building games – but at this point, I ran into another snag. The lighting wasn’t working – a practical problem as I didn’t know how lighting works in ARKit and picked up some useful code here.

directionalLight.light.color = .white
directionalLight.light.intensity = 500
directionalLight.light.isRealWorldProxy = true
directionalLight.shadow?.maximumDistance = 1
directionalLight.shadow?.depthBias = 4.0
directionalLight.orientation = simd_quatf(angle: Float(0).degrees2radians(),
axis: [0,1,0])

let lightAnchor = AnchorEntity(world: [0,1,0])

game state

Bonn – As we come to this part of the WWDC2019 presentation, I suspect they were running out of time. I think it’s because they left out an important side to explain how to integrate the entity/component code they explain with it. Worse, I was confused by the horrible naming of ModelEntity in the async code shown [changed in my code], In presentation, they call it “model”. a name that effectively means you need to refer to the model within the model model.model Not good, Apple.

Back in the real world, you have two players in chess, one white and one black, who take turns. So my custom component would be the color of the piece you just tapped on to make sure it’s the set you’re trying to move. I also wanted to assign a value to the individual pieces, a semantics that I could use later to decide who is winning or losing the game.

enum PieceColor: Codable 
case black
case white

struct PieceComponent: Component, Codable
var color: PieceColor
var value: Int

class ChessEntity: Entity, HasModel, HasCollision, HasPhysics
//public var model: ModelEntity!
public var piece: PieceComponent
get return components[PieceComponent.self] ?? PieceComponent(color: .white, value: 0)
set components[PieceComponent.self] = newValue

I add a new component to a custom class that I created to represent the chess pieces, with actions you can take on them. At this point I again wanted a minimum viable product.

extension ChessEntity 
func movePiece()
print("name \(")
self.transform.translation -= SIMD3(0,0,-0.1)

Code that you integrate with your existing routines, ie the loading of assets and the detection/action of tapping on a fragment. The only change you need to make to the load that setup the unit is two lines.

let piecePlayer = ChessEntity()
piecePlayer.model = entity.model

After placing the important pieces, I was ready to add action to the chess game. That’s more or less the action I got from the WWDC2019 presentation. code triggered by a simple UITapgestureRecognizer,

@objc func handleTap2(_ sender: UIGestureRecognizer? = nil) 
let tapLocation = sender?.location(in: arView)
if let piece = arView.entity(at: tapLocation!) as? ChessEntity


Now after everything is in place, I made the board and placed my chess pieces on it. I was ready for the next step.


And there you have it, a prototype of an augmented reality version of Chess, using not only RealityKit, but ARKit as well — the two feel almost inseparable.

Of course, there is still a lot to implement – in the presentation, they end up with a multiplayer version, this is a significant difference with SceneKit and for me it should be implemented. I want to try to do a little more with the moves this time than I did both this and last time; There may be some intelligence built into the game as well.

Finally, my thoughts then go back to when I started this journey in March 2022. Then I argued that Seenkit was not dead; It was just relaxing. It’s too early to say if I’ve changed my mind, but I’m struck by the similarities between RealityKit and SceneKit.

That said, searching around, I found this excellent piece by Tony Morales An article he wrote two years ago, comparing the two. I’ve since met with him and he told me that “most of my issues have been resolved” – which now looks and sounds like RealityKit has replaced SceneKit – I’ll let you know if I do this time next year Changed my mind when I did have a chance to really dig myself into it. Follow me on Medium for more posts on the topic.

Leave a Reply