Fun with code (since 2006)

A sneak peek: Concurrency with ActionScript Workers


06.11.12 Posted in Actionscript 3, Inside the Flash Player by

This week we just shipped Flash Player 11.3/AIR 3.3 with lots of cool features for mobile and desktop. But, we are already finalizing the last details on a brand new feature coming very soon, ActionScript Workers. Planned for the next Flash Player/AIR release, I am super excited about this feature. Here's why.

The concept of concurrency is kind of the holy grail for most ActionScript developers who are working on complex projects. How many times, have you had issues with the UI locking and having to distribute computations to frames like crazy? How many times did you have to optimize your algorithms like crazy to minimize the cost of execution and reduce the risk of UI locking? Yeah, you have been there.

Concurrency has always been popular in the Jira(290 votes), I remember having conversations over there with many developers following the status of this feature, I am sure some of you may think, well, better late than never. Again, the focus we have now on gaming or video, allows us to focus, so deliver valuable features like those much quicker.

See all the features we introduced between Flash Player 11 and 11.3, most of the biggest requests from ActionScript developers are getting delivered and we will not stop there :)

We started working on this feature last year, at that time, we were debating over how we would introduce concurrency to ActionScript developers, through a simple and safe way. We decided to go with a workers approach, cause it is a design most developers are today familiar with, which also can be API based rather than language based. We knew that we would not introduce a new keyword (for now) to leverage concurrency, so really workers made a lot of sense.

How does it work?

I will cover the feature in much more details when we ill have our first public beta, but here are some details about it. Like you would expect, you can create workers through a Worker API. The Worker takes as an input a SWF in the form of a ByteArray, which will be generated by the compiler and injected to create your Worker. Flash Builder will support this nicely and transparently, any other IDE you use will also support this when they implement concurrency support.

Before we start, let's cover two simple notions. In this article, we will refer to the UI worker (the one you always worked with in the past) as the primordial worker. It is still the entry point of your app, the primordial worker is never created by user code and is created automatically by the runtime, it is essentially your good old document class, extending Sprite at least.

So to create your child worker from the primordial worker, you will just write:

var worker:Worker = WorkerDomain.current.createWorker(Workers.EncoderWorker);

As you can see, the Worker is created through the help of the WorkerDomain object, that you can see as a factory. One and only WorkerDomain object exists for the entire runtime, and cannot be constructed by user code. Workers.EncoderWorker points here to our embedded worker using the [Embed tag].

At this point, our background logic is not executed, to trigger this, we simply use the start() API:

worker.start();

Now, the code you would have in the constructor in your background worker is started and waiting for the action. The Worker class has a state instance property to allow you to check at any time which state is your worker in. The different states are available through constants of the WorkerState class: NEW, RUNNING, TERMINATED

You could create as many workers was you want, but keep in mind that it is costly to create tons of them, each Worker is in fact a constrained virtual Flash Player instance, so it has a cost in memory (5-7mb). Ok, so now we need to communicate between workers. For this, please welcome the MessageChannel API. To communicate and send a message from a worker to another, you would create such a message channel like this:

var mainToBack:MessageChannel = Worker.current.createMessageChannel(worker);

The Worker.current call refers to the current worker (the primordial) worker, communicating with the background worker, passed here in parameter of the createMessageChannel API.

Now, we have our messaging channel in place, at any time, we can just call send() on the MessageChannel object to pass data to the background worker. But wait, we need a way to retrieve this message channel on the other side, so that we can communicate. For this, you will use the setSharedProperty API, which will put a reference to your message channel so that the child worker can grab the reference and listen to the messages coming in.

So, from the primordial worker you would write:

worker.setSharedProperty("mainToBack", mainToBack);

And in your child worker:

mainToBack = Worker.current.getSharedProperty("mainToBack");

Note that the String used here should be (to make things clean) something reflect the package of your program, I am using here something obvious to reflect the direction of the communication very clearly. So there we are! To pass data, we can now use our message channel pipeline. Also, remember that the setSharedProperty API can also be used to simply pass values to be used during the initialization of the Worker.

In your application you may want to send data like a String, a Number, an Array or other lightweight types like this, for this you can just use myChannel.send(), the protocol used behind the scenes is AMF3. You can also send custom classes, it will behave as a ByteArray.writeObject/readObject. You would then need to register your class with registerClassAlias. For the ones familiar with Flash Remoting or Flash Media Server, you guys get the idea, your data will be copied and serialized through the rules of AMF3. So no way to pass any DisplayObject between workers for instance, as it is not supported by the AMF3 format. Also, keep in mind, that most data types passed through MessageChannel are serialized (copied) at the exception of a few I will cover later in this article.

So let's say you want to pass an int to a worker, you would write:

mainToBack.send(100);

On the other side, you would simply write:

var value:uint = mainToBack.receive();

You would usually place the receive() call in the Event.CHANNEL_MESSAGE event handler dispatched by the MessageChannel object. Now, the receive() call has two flavors: Synchronous and asynchronous which is actually super useful. In some cases you may want to block and wait until the sending worker has sent the message, that's why there is a blockUntilReceived parameter on the receive() API. By default, it will not block.

You can also queue multiple messages:

mainToBack.send(100);
mainToBack.send(true);
mainToBack.send([10,20]);

On the other side, you can consume them by just checking the messageAvailable property on MessageChannel and consume each of them by keep calling receive().

while ( mainToBack.messageAvailable )
mainToBack.receive();

So now you may wonder, ok this is really cool but what if I have big amount of data to transfer, like, let's say a BitmapData, or more simply a ByteArray? Well, here comes shared memory to the rescue!

Shared memory

Using shared memory is the most efficient way to transfer data between the workers. Essentially, it is a simple ByteArray class, the one you know which now has a new magical property: ByteArray.shareable

To pass around a reference to your shared memory, you use the same MessageChannel.send() API, but when you pass a ByteArray, it is not serialized, just referenced. Once you have a shared bytearray, you can pass anything you want between the workers, nothing will be copied but shared. So let's say you want to pass the same primitives we sent earlier, you would just do:

sharedMemory.position = 0;
sharedMemory.writeUnsignedInt(100);

On the other side, you just retrieve that simply:

sharedMemory.position = 0;
var value:uint = sharedMemory.readUnsignedInt();

Of course, you can queue different things, like:

sharedMemory.position = 0;
sharedMemory.writeUnsignedInt(100);
sharedMemory.writesBytes(textures);

The variable textures would refer here to a ByteArray containing your compressed textures, and voila. You could pass them that way, simple, right?

Working with pixels

Knowing that workers would be used for image processing also, we also added a new API to BitmapData called copyPixelsToByteArray to work nicely with the concept of shared memory. This allows you to pass efficiently the pixels of a BitmapData to the shared memory, without having to have a temporary buffer you would allocate and copy the data from. That way, you can just write:

myBitmapData.copyPixelsToByteArray(myBitmapData.rect, sharedMemory);

And done. The pixels are transferred and accessible on the other side to be displayed or processed for any kind of work. I am sure you guys will find this one super useful ;)

Race conditions

As expected, if you have a shared memory between n workers, and that workers try to read or write to the same memory at the same time, you are on for troubles. To prevent this, you need to use mutual exclusion locks. That's why we will be introducing a new  flash.concurrent package with new classes : Mutex and Condition.

I will provide more details on those two guys later on with some examples. I suspect most developers will not necessarily need those, given that most developers will either just use 2 workers or multiple shared memory objects. You will really need to work with mutexes in the case where you need the multiple workers to read and write in the same memory concurrently, which can happen for some specific use cases!

What is accessible from a child Worker?

Not everything is accessible from a worker, obviously you cannot display things from a child worker and things like access to the Mouse, Camera and other UI APIs are also not available, but pretty much all the rest is. A display list is available, so that you can nest things, render them offscreen and work with that, for instance to rasterize things to a BitmapData. All network APIs are also supported if you need to load things in the background, super useful.

Demo

I finished porting a cool Alchemy lib (ShineMP3Encoder) from Cyril Diagne (flash-kikko), to perform MP3 encoding at runtime from a .wav file.  Today the lib performs in a single threaded mode affecting the UI thread. I ported it to leverage workers, each file encoding can now be parallelized using workers without ever impacting the UI thread. Here is a little video demonstrating the project:

I hope you guys are excited about it!



107 Responses to “A sneak peek: Concurrency with ActionScript Workers”

  1. wj says:

    I have been working on some major tooling for developing games for my current company and we pretty much had to stop it due to the way these workers are implemented.

    * Having multiple threads running is too memory intensive.

    * Bottlenecks in shared memory.

    * To awkward to use, its time consuming for any large project that actually needs multiple background workers that support a multitude of features. (In Java we would simply have each feature as a single class, nice and simple and understandable instead of messaging all over the place and we can queue them, schedule them in seq/par etc easily)

    * Java workers are way simpler.

    * Inability to access main domain definitions from a worker, we wanted to emulate some features of Unity 3D Annotated scripts that you can drag into an inspector, it was going to be real time etc. But required heavy use of AOP and ByteCode which locks the IDE for 4 to 5 seconds and that was unacceptable and we cant do it in threads cleanly. We knew if we went further we could have wait times of up to 10 seconds waiting for AOP.

    We are now investigating building our IDE in Objective-C or Java instead of AIR using workers which is a shame because we wanted to make it available to a wide audience but seeing as we all use macs in the office we are just going to build it for mac.

    And instead of a multi threaded framework for our game SDK we are simply going to load an XML file produced by our external tooling and the only threading support we will add to our SDK is for loading resources in the background.

    It’s a big shame really, we could have built such an amazing tool if we had access to real threading that was similar to the Java implementation, we cant spend twice the amt of time working on it, that’s just not worth it either.

    Anyway that’s just my 2 cents, I’m an avid fan of Flex and have been using it ever since 1.0 but it really needs to come to terms with the fact that every language has good/bad programmers and you cant hold our hands. Some of us want to build highly complex powerful tools but you are restricting us and forcing us onto other platforms.

    My company is now considering Unity 3D for all it’s game development as well which I’m not against however I much prefer building GUI’s in flex but I understand where management is coming from, using Unity would increase productivity / throughput and allow us to build more powerful games that run on every platform as well. I hope flash picks up the slack and gives us more powerful features we can actually put to good use aside from loading images and data in the background.

  2. Sean says:

    Any news on Workers on mobile?
    It’s the one thing Adobe need to add so we can develop high end games for mobile…

  3. Peter Herz says:

    While this is definitely cool and long overdue ‘native’ development for AS3.. I wonder for those that are looking to achieve the same thing but without requiring FP11.4 clients only.. and perhaps they want to do this but with mobile clients which this example doesn’t support.. then they use as3-commons async or greenthreads lib on google code btw.

  4. pol2095 says:

    still no mobile workers in air 3.6, any news on mobile Workers ?

  5. YopSolo says:

    no mobile workers in air 3.7 ?

  6. bank lån says:

    bank lån…

    A sneak peek: Concurrency with ActionScript Workers – ByteArray.org…

     
  7. sameer Hwety says:

    did worker support in Flash Builder 4.6 since i tried to add it but it doesn’t work ?
    If supported ,firstly , how to implement it in Flash Builder and using it . secondly , did it support for flex project or Action Scriptproject or both ?

Leave a Reply

Open Sort Options

Sort comments by:
  • * Applied after refresh

ByteArray.org
Fun with code since 2006.