Swift: Preloading an image and displaying it

Note: Source code (RemoteImage) available on Github

Yesterday, I explained how to use the NSURLConnection API to load a remote JSON file. Today I will show you how to use the same API to load a remote image. In fact, the NSURLConnection, similar to URLStream (in AS3) can load pretty much anything, basically it will download raw bytes, once loaded, you are free to do whatever you want with them. Serialize them to JSON, or pass them to a UIImage object, and there you go, you have your image ready to be displayed.

I commented each line which should be pretty straightforward:

import SpriteKit

class GameScene: SKScene {
    
    // our properties
    var bytes: NSMutableData?
    var totalBytes: Float64?
    let label: SKLabelNode = SKLabelNode(fontNamed: "Verdana")
    
    override func didMoveToView(view: SKView) {
        // we create our text label for the preloading infos
        label.position = CGPoint (x: 520, y: 380)
        addChild (label)
        
        // this is our remote end point (similar to URLRequest in AS3)
        let request = NSURLRequest(URL: NSURL(string: "https://s3.amazonaws.com/ooomf-com-files/yvDPJ8ZSmSVob7pRxIvU_IMG_40322.jpg"))
        
        // this is what creates the connection and dispatches the varios events to track progression, etc.
        let loader = NSURLConnection(request: request, delegate: self, startImmediately: true)
    }
    
    func connection(connection: NSURLConnection!, didReceiveResponse response: NSURLResponse!) {
        // we initialize our buffer
        self.bytes = NSMutableData()
        
        // we grab the total bytes expected to load
        totalBytes = Float64(response.expectedContentLength)
    }
    
    func connection(connection: NSURLConnection!, didReceiveData conData: NSData!) {
        // we append the bytes as they come in
        self.bytes!.appendData(conData)

        // we calculate our ratio
        // we divide the loaded bytes with the total bytes to get the ratio, we mulitply by 100
        // note that we floor the value
        var ratio = floor((Float64(self.bytes!.length) / totalBytes!) * 100)
        
        // we cast to Int to remove the decimal and concatenate with %
        self.label.text = String (Int(ratio)) + " %"
    }
    
    func connectionDidFinishLoading(connection: NSURLConnection!) {
        // we create a UIImage out of the completed bytes we loaded
        let imageBytes = UIImage(data: self.bytes)
        
        // we create a texture
        let texture = SKTexture(image: imageBytes)
        
        // then a sprite
        let sprite = SKSpriteNode(texture: texture)
        
        // we calculate the ratio so that our image can fit in the canvas size and be scaled appropriately
        var scalingRatio = min (self.view.bounds.width/sprite.size.width, self.view.bounds.height/sprite.size.height)
        
        // we apply the scaling
        sprite.xScale = scalingRatio
        sprite.yScale = scalingRatio
        
        // we position our image
        sprite.position = CGPoint (x: 510, y: 380)
        
        // we remove the percentage label
        label.removeFromParent()
        
        // we add our final image to the display list
        addChild(sprite)
    }
    
    override func update(currentTime: CFTimeInterval) {
        /* Called before each frame is rendered */
    }
}

During preload, this gives us the following:

Preloading

Once loading is complete, our image is displayed:

Loaded image

I hope this code might be helpful to you guys!

Swift: Loading and parsing a remote JSON file

Note: Source code (RemoteJSON) available on Github

Loading something remotely is a pretty simple task in Flash. We can use the URLStream or URLLoader object, a URL is wrapped inside a URLRequest object, we call the load() API and we are done.

So I was wondering what it looks like in Swift and specifically loading a JSON file, which is a common thing to do today, so here is a little snippet for that.

First, we use the NSURLRequest (not that different right?) object to specify our remote URL:

// this is our remote end point (similar to URLRequest in AS3)
let request = NSURLRequest(URL: NSURL(string: "http://bytearray.org/wp-content/projects/json/colors.json"))

Now, we need to use that object to connect over there, this is where the NSURLConnection comes into play, think of it as the equivalent of the URLStream object:

// this is what creates the connection and dispatches the varios events to track progression, etc.
let loader = NSURLConnection(request: request, delegate: self, startImmediately: true)

As a first parameter, we pass our NSURLRequest object, then we pass our current class as a delegate hander, and we ask that the request starts immediately. We could have passed on that, and call the NSURLConnection.start() API later.

Our data is now coming in, and we need to save it. With URLStream, the data would be gathered internally and stored, and calls to the URLRequest object directly would let you read the data, just like you would do with a Socket connection. In Cocoa with Swift, it is a little bit different, we need to create our buffer to store the data, then through the progress event, write the bytes to it:

func connection(connection: NSURLConnection!, didReceiveData conData: NSData!) {
     self.bytes?.appendData(conData)
}

We are using the optional operator cause we are initializing our property later on, when we receive the first response from the server:

func connection(didReceiveResponse: NSURLConnection!, didReceiveResponse response: NSURLResponse!) {
    self.bytes = NSMutableData()
}

When data is done loading, we can parse it:

func connectionDidFinishLoading(connection: NSURLConnection!) {
        
      // we serialize our bytes back to the original JSON structure
      let jsonResult: Dictionary = NSJSONSerialization.JSONObjectWithData(self.bytes, options: NSJSONReadingOptions.MutableContainers, error: nil) as Dictionary<String, AnyObject>
        
      // we grab the colorsArray element
      let results: NSArray = jsonResult["colorsArray"] as NSArray

      // we iterate over each element of the colorsArray array
      for item in results {
          // we convert each key to a String
          var name: String = item["colorName"] as String
          var color: String = item["hexValue"] as String
          println("\(name): \(color)")
      }
}

Note that we are casting our result to a Dictionary that we can query later on to extract our colorsArray key. This code above gives us in the following output:

red: #f00
green: #0f0
blue: #00f
cyan: #0ff
magenta: #f0f
yellow: #ff0
black: #000

Note that we did not specify listeners to the various callbacks that are being called during loading and when loading is complete. This is done implicitly when we passed our current instance (self) to the delegate property (when we create the NSURLConnection object). The delegate functions are called because they follow a specific naming convention. Have a look in the final complete code:

import SpriteKit

class GameScene: SKScene {
   
    var bytes: NSMutableData?
    
    override func didMoveToView(view: SKView) {
        
        // this is our remote end point (similar to URLRequest in AS3)
        let request = NSURLRequest(URL: NSURL(string: "http://bytearray.org/wp-content/projects/json/colors.json"))
        
        // this is what creates the connection and dispatches the varios events to track progression, etc.
        let loader = NSURLConnection(request: request, delegate: self, startImmediately: true)
    }
    
    func connection(didReceiveResponse: NSURLConnection!, didReceiveResponse response: NSURLResponse!) {
       self.bytes = NSMutableData()
    }
    
    func connection(connection: NSURLConnection!, didReceiveData conData: NSData!) {
        self.bytes?.appendData(conData)
    }
    
    func connectionDidFinishLoading(connection: NSURLConnection!) {
        
        // we serialize our bytes back to the original JSON structure
        let jsonResult: Dictionary = NSJSONSerialization.JSONObjectWithData(self.bytes, options: NSJSONReadingOptions.MutableContainers, error: nil) as Dictionary<String, AnyObject>
        
        // we grab the colorsArray element
        let results: NSArray = jsonResult["colorsArray"] as NSArray

        // we iterate over each element of the colorsArray array
        for item in results {
            // we convert each key to a String
            var name: String = item["colorName"] as String
            var color: String = item["hexValue"] as String
            println("\(name): \(color)")
        }
    }

    override func update(currentTime: CFTimeInterval) {
        /* Called before each frame is rendered */
    }
}

The didReceiveResponse and didReceiveData parameters allow us to specify through the method signature, which method is assigned to each event as a handler. Pretty succinct and efficient. The only exception is the connectionDidFinishLoading handler that needs to use that name, and that's it, we are done!

Swift: ARC vs Flash GC

In ActionScript 3, memory is managed through the help of a garbage collector that allocates and deallocate objects through the application lifecycle. The garbage collector (GC) allocates memory when a new object is created, scans the objects graph periodically, detects unreferenced objects and deallocated them, pretty useful. AS3 is not the only language that relies on garbage collection, C# with Mono, JavaScript or Java all rely on garbage collection. On paper, it sounds great, but any developer who has developed more complex content on a GC based platform will tell you that it's not all so blue. Even though GC makes the developer's life easy at first, ActionScript 3 developers worst enemy today is actually the GC, so why is that?

Unpredictability

First, garbage collection is completely unpredictable, I remember when teaching ActionScript 3 to students, the idea that objects would be deallocated "at some point" in time, but nobody knows when, was something pretty hard to grasp. Actually, it was even possible that objects would never get deallocated if the garbage-collector never decided to kick-in. So how do you test this? In ActionScript 3, to test/profile an application, it is possible to trigger the GC from the Flash Builder profiler, but also from AS3 with the System.gc() API or even better with Adobe Scout, that also provides information on who is eligible or who just got deallocated.

In the code below, we set the sprite reference to null, note that this does not trigger anything, our sprite is still alive:

import flash.display.Sprite;

var mySprite:Sprite = new Sprite();

mySprite.addEventListener ( Event.ENTER_FRAME, onFrame );

function onFrame ( e: Event ):void
{
	trace ('I am alive!');
}

// we dereference the object
// collection is not triggered, sprite is still alive and talking
mySprite = null;

At this point, our sprite is eligible garbage collection, but still remains in memory and still dispatches the Event.ENTER_FRAME. To test if our sprite will eventually be garbage collected, we can trigger the GC using the System.gc() API:

import flash.display.Sprite;

var mySprite:Sprite = new Sprite();

mySprite.addEventListener ( Event.ENTER_FRAME, onFrame );

function onFrame ( e: Event ):void
{
	trace ('I am alive!');
}

// we dereference the object
// collection is not triggered, sprite is still alive and talking
mySprite = null;

// collection is triggered, object is killed
System.gc()

Remember that the System.gc() API is a debug-only feature, so you cannot rely on it in production. This GC unpredictability can be pretty sneaky. Typically, you don't want the garbage collection to happen in the middle of something, in a game, where best performance is crucial, you don't want garbage collection to happen right in the middle of the game, bur rather before the new level gets loaded. In other words, at a time where the experience/performance is not impacted.

In Flash Player 11, we introduced a new API System.pauseForGCIfCollectionImminent() which helped developers influence when the garbage collection would kick-in. You still could not control the GC directly, but it is was an improvement.

Synchronous (UI lock)

The reason why you don't want garbage collection to happen at moments you don't control is because GC in Flash happens on the UI thread, therefore locks the UI when collection happens. The more complex your scene becomes, and the bigger the graphs are, the longer the pause will be. In a game, this is a showstopper, because UI lock ruins the experience and frustrates users.

That's why AS3 developers have developed workarounds over the years to prevent the GC from being triggered, object pools being one of the strategy. The idea behind object pooling is that instead of allocating new objects constantly and pressure the GC, a set of objects are allocated during app initialization and once objects are done doing their tasks, they are placed back inside a pool for later reuse. Keep in mind that this will do the job, but will consume more memory, as objects never get deallocated, you win on the performance side, but lose on memory footprint. You can find more details about object pooling here.

With Swift?

In Swift, thanks to ARC (Automatic reference counting) you also don't need to manage memory manually by allocating memory and releasing objects like with Objective-C before ARC was introduced. ARC counts the number of references pointing to objects and when the number of references reaches zero, it kills them. Pretty similar to ActionScript 3 you may say, but with a few notable differences.

Compile time vs runtime

In AS3, all this GC work happens at runtime, the bytecode generated by the ActionScript 3 compiler does not emit any specific calls to allocate or release memory. If you were to decompile the assembly code generated by the Swift compiler, you would see the calls to allocate and release objects, like if you had done it manually, in Swift, the compiler does all the work for you.

Synchronous deallocation

In AS3, like we have seen before, setting the last remaining reference to an object to null, won't kill the object. It will make it eligible for garbage collection. In Swift, it will immediately deallocate the object synchronously, and that is a big difference. You can track initialization and de-initialization through the use of the init and deinit methods:

class Hero {
    
    let name: String
    
    init ( name: String ) {
        self.name = name
        println ("\(self.name) got initialized")
    }
    
    deinit {
        println ("\(self.name) got deinitialized")
    }
}

// we create our hero
// note the use of the optional (?) operator
// using this operator allows the var bob to be set to nil
var bob: Hero? = Hero(name: "Bob")
        
// we set the only reference to nil (equivalent of null)
// the object is immediately destroyed and the deinit method is called
bob = nil

If we run our application, we see in the output window:

Bob got initialized
Bob got deinitialized

Because the developer has full control over when the objects are deallocated, objects are killed sooner, optimizing for memory consumption (there is no pool of eligible objects in memory waiting to be disposed). It is also a more incremental approach that prevents the UI from locking. Most GC, on most platforms, as beautiful and complex as they are, will always impact the UI thread.

Circular references (retain cycles)

In AS3, if two objects were unreachable from the roots of the application (Stage, Loader), but were still referencing each other, they would still be garbage collected.

In Swift, if two objects are unreachable from the roots of your application, but are still referencing each other, you have what is called a circular reference or retain cycle and these two objects will never get deallocated and probably cause a memory leak. The GC in Flash Player/AIR solved this through the combination of deferred reference counting and conservative mark-and-sweep. The mark and sweep piece is what handles circular references (retain cycles) and that is a big advantage of garbage collection in Flash Player/AIR.

To deal with this in Swift, you would use weak references to express that if the last reference to an object is weak, the object should still be deallocated. Swift also introduces the concept of unowned references, which are non optional. This brings more granularity in object dependencies, you can read more about it here.

I hope you enjoyed this quick overview of differences between the Flash memory model (GC) and Swift (ARC).

Swift: Overflowing

As you guys know, I always loved playing with binary stuff, and one of the frustrating thing in ActionScript has always been the lack of more granular data types. In ActionScript 3, whatever type you use, it will either occupy 64 or 32 bits in memory, even for a simple boolean. Not super optimized. So if you were to work with a byte (which is 8-bit), the smallest data type you could use to store it would be an int (32-bit signed integer) or a uint (32-bit unsigned integer).

So what's up with overflowing? Well, I first encountered that problem when working on a Intel8080 CPU emulator in AS3. Every CPU register for the Intel8080 is a 8-bit register. Because there is no byte type equivalent (8-bit integer) in AS3, as I said earlier, you are left with int or uint, therefore you are using 32-bit for each 8-bit register, meaning that no error will be triggered if you try to store more than 8-bits on the register. This is what overflow means, when you try to store more data than what the type can hold.

So in AS3, I ended up spending hours trying to understand what was wrong. The code below illustrates the idea:

// 255 is the max value a unsigned byte can hold
// but because I am using a uint, it will overflow
var myRegister: uint = 255;

// if I add 1 to it
myRegister += 1;

// I end up with 256 which is 100000000 in binary
// outputs: 100000000
trace ( myRegister.toString ( 2 ) );

Because a uint can hold 32 bits, by adding I end up with 256, represented as 9 bits (100000000). Makes sense right?

Makes absolute sense, but in the context of a register that is never supposed to go higher than 255, it is a problem. After hours of debugging, I thought that masking the registers to only preserve the rightmost 8 bits could help. That was indeed the problem, but it took me forever to discover that. I recall my buddy Claus had the same issue when working on FC64 (a Commodore 64 emulator in AS3). So I ended up masking pretty much all operations on registers. Here is an example of how it works:

// 255 is the max value a unsigned byte can hold
// but because I am using a uint, it will overflow
var myRegister: uint = 255;

// if I add 1 to it
myRegister += 1;

// I end up with 256 which is 100000000 in binary
// outputs: 100000000
trace ( myRegister.toString ( 2 ) );

// we preserve only the rightmost 8 bits
myRegister &= 0xFF;

// outputs: 0
trace ( myRegister.toString ( 2 ) );

In the CPU emulator, it looked like this, see the first line:

protected function PerformCompSub(inValue:int):void
{
    var value:int = (A - inValue) & 0xFF;
    CARRY = int(((value >= A) && (inValue)));
    HALFCARRY = ((A ^ inValue ^ value) & 0x10);
    ZERO = int((value == 0));
    SIGN = (value & 128);
}

So what is great about Swift? Well, first, Swift provides a more granular data type, like Int8 or UInt8:

var unsignedByte: UInt8 = UInt8.max
var signedByte: Int8 = Int8.max

The first advantage, is that these two variables will only occupy 8 bit in memory, so only what's required. That is a good first optimization. Second benefit, Swift leverages type inference, so you don't need to explicitly specify types. If you don't, like with the code below, the compiler will automatically infer the most efficient type, in that case (UInt8 and Int8):

var unsignedByte = UInt8.max
var signedByte = Int8.max

Third benefit, if you try to add 1 to this variable, because you are already occupying 8 bits, adding 1 will cause a compile time error, so the code below:

var unsignedByte = UInt8.max
unsignedByte += 1

Will cause the following compiler error:

Arithmetic operation '255 + 1' (on type 'UInt8') results in an overflow

Pretty cool right? In AS3, it would overflow silently and that can be sneaky. At runtime, same safety, it would trigger a runtime exception and that would have saved me hours of debugging in AS3. But in some scenarios you may have bigger values but still want the overflowing to happen silently and perform the masking operation implicitly. I can do that in Swift by using the &+ operator:

// our unsigned byte set to 255
var unsignedByte = UInt8.max

// we add 1 to 255, using the &+ operator instead of + to allow overflow
unsignedByte = unsignedByte &+ 1

// outputs: 0
println(unsignedByte)

More granular data types and advanced operators are very welcome in Swift, I will probably port the Intel8080 emulator in Swift and talk about it here. In the mean time, you can use the ampersand (&) before all common operators: +, -, /, * to allow overflowing and underflowing. For more details about this operator and this topic, check the Advanced operator section in the Swift documentation.

Swift: Drawing text on bitmap

Note: Source code (DrawingText) available on Github

In the last article, we covered the basics of bitmap drawing with Swift and the Quartz/Core Image APIs. As I was playing with that, I wondered how text could be drawn on a bitmap. In Flash, the BitmapData API does not have an API for drawing text, the workaround was to use a TextField offscreen that would be passed to the BitmapData.draw API.

On iOS, there is also no API to directly do that on an image, the idea is to rely on the graphics context, draw our initial image on the graphics context, then draw the text on the context too, then grab the resulted composited image using our best friend UIGraphicsGetImageFromCurrentImageContext.

Here is below what we are going to produce as a result:

copyright

As you can see, we will add at the bottom right of the image a small copyright message, composited onto the image.

This idea was brought up on this thread on Stackoverflow, so here we go with the Swift code to achieve this:

import SpriteKit

class GameScene: SKScene {
    override func didMoveToView(view: SKView) {

        // we pick the font we want to use
        let font = UIFont(name: "Arial", size: 18)
        
        // our string, note that we use here NSString instead of String that has more APIs like drawInRect and size
        // it is preferred to use Swift native rypes like String but for now String has a limited API surface
        let text: NSString = "Copyright © - Thibault Imbert"
        
        // we reference our image (path)
        let data = NSData (contentsOfFile: "/Users/timbert/Documents/Ayden.jpg")
        
        // we create a UIImage out of it
        let image = UIImage(data: data)
        
        // our rectangle for the drawing size
        let rect = CGRectMake(0, 0, image.size.width, image.size.height)
        
        // we create our graphics context at the size of our image
        UIGraphicsBeginImageContextWithOptions(CGSize(width: rect.width, height: rect.height), true, 0)
        
        // we retrieve it
        let context = UIGraphicsGetCurrentContext()
        
        // we set our color to white (this will be the text color)
        CGContextSetFillColorWithColor(context, CGColorCreateGenericRGB(1, 1, 1, 1))
        
        // we draw our image to the graphics context
        image.drawInRect(rect)
        
        // a dictionary informing about the font used, required by sizeWithAttributes to query the text size
        let attr = [NSFontAttributeName: font]

        // the size of our text
        let size = text.sizeWithAttributes(attr)
        
        // the rect for the drawing position of our copyright text message
        let rectText = CGRectMake(image.size.width-size.width, image.size.height-(size.height+4), image.size.width-(size.width+4), image.size.height)
        
        // we draw the text on the graphics context, with our font
        text.drawInRect(rectText, withFont: font)
        
        // we grab a UIImage from the graphics context
        let newImage = UIGraphicsGetImageFromCurrentImageContext();
        
        // we remove our bitmap from the stack
        UIGraphicsEndImageContext();
        
        // we create a texture, pass the UIImage
        var texture = SKTexture(image: newImage)
        
        // wrap it inside a sprite node
        var sprite = SKSpriteNode(texture:texture)
        
        // we scale it a bit
        sprite.setScale(0.5);
        
        // we position it
        sprite.position = CGPoint (x: 510, y: 300)
        
        // let's display it
        self.addChild(sprite)
    }
    
    override func touchesBegan(touches: NSSet, withEvent event: UIEvent) {
    }
   
    override func update(currentTime: CFTimeInterval) {
        /* Called before each frame is rendered */
    }
}

Note that in this example, we are relying on the NSString object, which is not a Swift native type. For now, the String object in Swift has a pretty limited API surface compared to the NSString object, with convenient APIs like sizeInFont or drawInRect. Hopefully the native String type in Swift will be better equipped in the future.