Here is MicRecorder, which handles internally the ByteArray work with the Event.SAMPLE_DATA event dispatched by the Microphone object introduced in Flash Player 10.1 and the WAV packaging. The WAV encoder is bundled but any other encoder could be used in a near future

Here is a little demo :

To record audio from the Microphone in your application, just use those few lines :

// volume in the final WAV file will be downsampled to 50%
var volume:Number = .5;
// we create the WAV encoder to be used by MicRecorder
var wavEncoder:WaveEncoder = new WaveEncoder( volume );
// we create the MicRecorder object which does the job
var recorder:MicRecorder = new MicRecorder( wavEncoder );
// starts recording
recorder.record();
// stop recording
recorder.stop();

When recording starts a RecordingEvent.RECORDING event is dispatched giving infos about time. When recording it stopped an Event.COMPLETE is dispatched allowing you to retrieve the Micorder.output bytes and save the audio stream (in this case as a WAV) using a simple FileReference object :

recorder.addEventListener(RecordingEvent.RECORDING, onRecording);
recorder.addEventListener(Event.COMPLETE, onRecordComplete);

private function onRecording(event:RecordingEvent):void
{
     trace ( event.time );
}

private function onRecordComplete(event:Event):void
{
     fileReference.save ( recorder.output, "recording.wav" );
}

You can also replay what has been recorded by passing the raw WAV file to the WavSound object from the nice as3wavsound library :

private function onRecordComplete(event:Event):void
{
     var player:WavSound = new WavSound(recorder.output);
     player.play();
}

The MicRecorder object relies by default on the default Microphone device available, but you can pass any Microphone instance as replacement when creating the MicRecorder object :

// a specific Microphone instance can be passed
var recorder:MicRecorder = new MicRecorder( wavEncoder, microphoneDevice );

You can download everything here.

If you are into 3d development for games, augmented reality or just interactive stuff like websites, you just can't miss the session entitled Flash Player 3D future scheduled for Max 2010 scheduled on October 27 at 11:00AM in room 503. Sebastian Marketsmueller (Flash Player engineer) will deep dive into the next generation 3D API coming in a future version of the Flash Player.

Now you may wonder, what does this means, what kind of 3D are we talking about ?

What kind of API ? True textured z-buffered triangles ? GPU acceleration ? Even better ? What I can say is forget what you have seen before, it is going to be big

When this will be available ?

We will share plans with you at Max during this session, I tell you, some serious stuff is coming for 3D developers.

If you are also curious about the inner details of the Flash Player renderer, Lee Thomason (Flash Player architect) will delve into the details of the Flash Player renderer, and show how to optimize the rendering performance of your applications. Lee will cover mechanisms like the display list, text rendering, shaders, GPU hardware acceleration, and exclusive features coming in a future version of Flash Player (hehe).

After this session, Flash Player rendering will no longer be a mystery for you. This track called Deep Dive into Flash Player Rendering is scheduled on October 27 at 09:30AM in room 511A.

I will be happy to meet you there at Max in Los Angeles and talk about our future plans and also get your feedbacks about the player around a fresh beer. I will post further details about all this in the following weeks.

If you haven't checked all the sessions available for Max this year check the online scheduler.

This session sums up the two white papers I wrote lately on this topic. So if you did not read the papers or just want to share that with colleagues who did not want to read all that, this can be a good solution

Best Practices in optimizing content for Flash Player 10.1 from Thibault Imbert on Vimeo.

You can download the paper here (bottom of the post). Or direct link here.
If you guys can review it that would be great, if you find some errors, any topic por example to be added or changed.

By working on this whitepaper I thought it could be funny using the new XFL file format introduced in Flash Pro to actually audit Flash projects and highlight bad graphical practices. I started a little native tool called "XFL Gourmet" which does that for you, you just pass it an XFL project, it will parse it and output the result. Of course this is pretty raw for the moment, but with this XFL format, anything can be detected, even a non-optimized vector shape could be detected.

Here is a snapshot of the output on a project using a lof of bad things, like transformed and animated bitmap caching/filters or blend modes :

I will keep working on it and make it available with the source code so that you guys can modify it and code your own rules.

What is really being called ? And the time it took for it.

This would allow us to provide you a rendering profiler perfect for fine tuning. It could generate a heat map of the rendering operations to see which operation internally is costing too much CPU cycles. A perfect way to detect also some side-effects which were not supposed to be there but where finally injected through the development of the application. A filter, some nested bitmap caching, or some blend mode or even a transparent bitmap used where it should not.

To get those infos you can use Shark on MacOS or VTune on Windows and Linux. The idea is simple, you run it, start collecting samples and start your SWF.  Once you are done, you stop collecting and it gives you an output. Without the Flash Player with the debug symbols you will not see anything relevant, just symbol adresses but nothing humanly readable. Like the following image :

You can already see that some processes are costing more time than others. With the debug symbols (included to a special version of the debugging player), you can see all the internals. So let's say I use a BitmapData object to paint pixels in my application. The following output, shows me clearly that almost 18% of the processing time was taken by the BitmapData.setPixel32 which calls internally the AddDirtyRect API from SurfaceImage and the internal core BitmapDataObject class and its setPixel32 API :

By the way, you can see that the AVM spend some time leveraging SSE2 instructions through the AVMCore class. Same thing, let's say you have to audit a SWF to see if some bad practices are used there. In the following SWF, we see a PixelBlitThread which is the API used for bitmap caching, so there is a lot of chances that you would take a look at the code of this SWF to see if the bitmap caching is used properly and if memory is ok :

Pure jewel for advanced optimizations. We are thinking about approaches to make this available (not in this raw-technical format) to every Flash developer. Do not hesitate to share ideas.

I'm very proud to announce that I will join the SID LEE team in few weeks as an Head of Creative Technology (Europe). For those who don't now SID LEE , you can visit www.sidlee.com.

I will be based at SID LEE Paris and will work closely with the others international offices : Montreal & Amsterdam.

With new mobiles, touch devices, GPS, and all the innovations to come, the digital playground become more & more amazing and I'm very happy to play now with SID LEE artisans.

SID LEE is very invested & open-minded about sponsoring R&D / innovation, so, I will continue to post & share some open source cool shit on bytearray

Stay tuned.

Didier.

The API is called appendBytes on the NetStream class used in conjunction with the NetStreamAppendBytesAction class. It allows you to inject a ByteArray to the NetStream object and play the video. This API was designed originally for HTTP streaming, allowing forward seeking into HTTP delivered-video, but if you just load a ByteArray locally  through the FileReference API, voilà, you got local preview of video files without server round trip. If you read the documentation you'll notice that the API only works for now with FLV header-based files, but all codecs are supported (Sorenson, H.264, On2 VP6 and even AAC audio).

Here is a little demo which illustrates the API in the Flash Player 10.1 (make sure you are using the Flash Player 10.1 RC4 available on labs.adobe.com) :

The API is really simple :

// retrieve the FLV stream
var bytes:ByteArray = event.currentTarget.data;
// put the NetStream class into Data Generation mode
netstream.play(null);
// before appending new bytes, reset the position to the beginning
netstream.appendBytesAction(NetStreamAppendBytesAction.RESET_BEGIN);
// append the FLV video bytes
netstream.appendBytes(bytes);

This will allow new kind of apps and remove one more limitation that we had in the past

So what do I mean by saying optimized ?

As you can imagine, it is mainly related to CPU usage and memory, and how simple ads can sometimes require more CPU cycles than they should. So, I decided to browse the web for a day and gathered as much ads as possible, I had a large choice, and I decided to save different kind of ads, made of video, bitmaps, manually or programmatically animated. Let's just put this out there, the intention of this post is not to criticize people producing ads but better understand what kind of pitfalls designers and developers tend to fall into when producing ads and why. More than that, I am sure that most of these bad practices could be adressed by the tools by giving more feedback on how to produce optimized content.

First thing people tend to forget is how a shape can be simplified through tools like the Simplify tool in Adobe Illustrator or just through the Smooth tool in Flash Pro. A lot of control points can be removed from paths and dramatically reduce the size of the final SWF and improve rendering performance. Another issue, in order to bypass the size limitations and the usage of bitmaps, designers use dynamic filters for any effect they need. As you know it, filters are not CPU-friendly when overused, especially when nested and with attributes changing over frames. Each frame the filter has to be recalculated and this can slow down rendering performance and require extra CPU cycles. Same for transparency effects which can be avoided or truly removed by using opaque DisplayObjects (opaqueBackground).

Whatever coding environnement you use, assets may still be created inside of Flash, exported to SWC or SWF and then linked inside your project. For ads, most of the work is done in Flash Pro only, cause code used is not complex and can fit on the timeline only.

I am wondering what kind of things could be added to Flash Pro for for instance, to help designers and even developers to produce optimized assets in general.

Here are some features I thought about :

• Show Redraw Regions

This should be enabled by default in the IDE (when playing animations on the Stage) and animators, designers, developers, would better understand what is wrong and costing CPU cycles for nothing. Ex : DisplayObjects hidden by others and still being renderered and not deactivated.

• Expose the opaqueBackground feature in Flash Pro IDE just like cache as bitmap.

Sometimes transparency is not required, and designers could disable it by assigning a background color to any DisplayObject. This would make it easier for the Flash Player.

• Native Bitmap Rasterization

You select a MovieClip and rasterize each frame as a bitmap, produce bigger SWF's in some cases, but blazing fast in terms of rendering. This could be performed at runtime by the player too, but a first step would be doing it at authortime.

• Graphical effects presets for publishing

I create and save a publishing profile for Nike and I say, disable all the filters, the transparency effects and instantly see performance differences by checking and unchecking effects between different publishing.

• History Panel 2.0

The History panel in Flash CS5 would be enhanced by showing more than the size and time history but also average framerate, CPU and memory usage.

• Optimization Perspective and size limit

The tool could have an optimization perspective, which would audit your graphics and let you know about potential graphics optimizations to reach a specific framerate and size, with text rendering, fonts, control points reduction, etc.). It would allow you to specify a size limit and the tool would generate a report by saying you can compress that bitmap more, you can use device fonts here, etc.

• Having a "Warning Mode" for graphics.

The tool shows you a heat map of the graphics in your app, showing where rendering time is spent. Warn about what is costing a lot in memory and CPU like nested filters, animated filters, bad usage of cacheAsBitmap, non-simplified shapes, etc. This should be pretty cool in the profiler for Flash Builder too.

For the Flash Player, maybe an HTML tag (stricter than the hasPriority tag in Flash Player 10.1) which would restrict the SWF being played to consume a specific amount of memory and CPU, otherwise throttle down the SWF. Another solution would be automatically removing specific effects when the framerate and performance is critical ? Is that something you would prefer ?

Let me know if you think any topic should be covered in the whitepaper, I would be happy sharing your ideas about all that.

Download the whitepaper here.

With some delay, due to many projects lately, here is the first chapter covering the basics of binary data in Flash. This first chapter covers all you need to know to get started, from the roots of binary to counting and thinking in binary with bitwise operators in ActionScript 3 and your first steps with the ByteArray class. Each concept is illustrated with real world examples so that you can really see and understand its usage. Those who read "Pratique d'ActionScript 3" will recognize some parts of the ByteArray chapter. I completely refactored it, translated it and included more technical details.

This chapter illustrates that the ByteArray class can be used everyday even for simple stuff you did not imagine. I will post each chapter when they are available on the book page here. Grab this first one and let me know if you have any comments (errata or stuff you'd to like to add).

I am starting now the second chapter entitled "Everyday bytes" focused on using binary stuff on everyday life as an ActionScript 3 developer, from handling binary stream (XML, fonts, etc.) to RSL and runtime compression. This chapter should cover also the great AS3SWF and FZip projects from Claus and other funky byte-oriented libraries like AlivePDF, or FLVSlicer and more.

Download

As I was saying during my interviews, Flash Player is a lot more than a simple video player, you can have 4 people using the Flash Player differently, from cartoons, to games, to websites (with video streaming) with ActionScript on top of it to make all that interactive to pure ActionScript projects without any external asset. This is what we saw again at the FITC this week in Amsterdam. So many people, using the same technology but differently. Erik Natzke will create amazing interactive design experiences and paintings, Chris Georgenes will animate a cute little monster in a cartoon, and advanced ActionScript coders will do crazy frameworks, work on CPU emulation or runtime audio tools or even complex data vizualisation applications for bank companies. To me, this is the power of the Flash technology, it talks to anyone, from design to code. I even met a team this year using Flash Professionnal, who did not even know what was a SWF, they were using the tool to animate and then export high definition PNG's to target HD TV's for broadcasting. The day after, I could meet a team who did not even know what was Flash Professional, and thought that Flash was just the Flash Player.

Oh there is a tool called Flash Pro ? Wow, did not know that

More generally, I also want the Flash Player to perform beautifully everywhere, and we are already stepping into this (See Tinic's post related to CoreAnimation). So many things come into my mind when I think about new stuff or improvements for an animator, an interactive designer or an ActionScript developer. I look forward to work with this incredible team and help driving the future of the Flash Player. Damn, I am excited!