Flash Player 11.2 and AIR 3.2 beta builds hidden gems by Thibault Imbert

I want to share with you some hidden gems available in Flash Player 11.2 and AIR 3.2, that we just released for beta this week. Those were not mentioned in the release notes, cause they will not be part of 11.2 and AIR 3.2, but will be part of a next release. But that's cool, quickly grab the beta builds, that way you can start playing with those APIs, just use -swf-version=15 to unlock them ;)

You won't find any docs on those, so here we go, let's start with the new API that we added on StageVideo : StageVideo.attachCamera(), it says it all. You can now attach your camera to a StageVideo object, to use very little CPU when displaying the camera stream:

public class StageVideo extends EventDispatcher
{
     public native function attachCamera(theCamera:Camera):void;
}

For the image processing lovers, we added Camera.drawToBitmapData, Camera.copyToByteArray, Camera.copyToVector. No need to manually BitmapData.draw a video object to get the camera pixels, and same for the pixels as a ByteArray or Vector. Notice that those should have very low impact on the GC. Rather than creating new buffers for each call, you get to provide the buffer to fill, and just call the API :

public final class Camera extends EventDispatcher
{
     public native function drawToBitmapData( destination:BitmapData ):void;
     public native function copyToByteArray( rect:Rectangle, destination:ByteArray ):void;
     public native function copyToVector( rect:Rectangle, destination:Vector. ):void
}

We also added an event, informing you about the availability of a new video frame, this way you can execute your image processing logic in this event rather than Event.ENTER_FRAME, way more efficient:

public class Event
{
     public static const VIDEO_FRAME:String = "videoFrame";
}

For those who recall this conversation on Twitter, some of these additions were added after a conversation with Mario and Eugene, hope you like these guys! :)

Inspirit - getFrameBytes

Comments (52)

  1. thienhaflash wrote::

    Hi Thibault, thanks for the information

    Friday, October 28, 2011 at 5:20 am #
  2. sascha wrote::

    So wait … You have hidden gems in FP 11.2 and AIR 3.2 but they are not yet part of 11.2 and 3.2?? But they will be part of the next release??? Excuse me, but you lost me there! swf version 15? What about swf version 14?

    Is it mandatory that one must have read the Twitter thread to understand the logic behind this?

    Ok so raise your hand anyone of the 1000+ people who are reading this who are interested in new webcam features? … I see two hands! Wow!
    And how many of you are using your webcam for more than just an occasional gimmick? … I see one hand! Waaw! That’s great!

    Seriously, can Adobe move onwards and add features that are important and long requested? you know, things like mouse locking so we can actually create controllable games with Stage3D!

    Friday, October 28, 2011 at 5:27 am #
  3. Well, aside from Saschas reasonable response that its hard to understand how you choose what to implement: Wasn’t the big point of @inspirit’s remark to get each channel separately?

    Friday, October 28, 2011 at 5:50 am #
  4. Héctor wrote::

    There is a big market in apps that use the camera, so these additions are welcome.
    However, if no bugs are reported on these features, I think these features should be in Flash 11.2.
    Any sneak preview on what Flash 11.1 will have?

    Friday, October 28, 2011 at 8:33 am #
  5. Thibault Imbert wrote::

    Hi Martin,

    Direct extraction is the fastest path. Doing channel extraction would slow it down a lot, maybe not be worth adding it natively. Could be done in AS3.

    Thibault

    Friday, October 28, 2011 at 8:36 am #
  6. Ludo wrote::

    Mmm… strange…

    TypeError: Error #2007, null value in VIDEO_FRAME:

    public static const VIDEO_FRAME : String;

    Perhaps must be:

    public static const VIDEO_FRAME : String = “videoframe”.

    Anyway, congrats. :)

    Anyway to use this VIDEO_FRAME in normal videos (or stage videos)? With (oh my god!) a number of frame variable?

    Friday, October 28, 2011 at 9:52 am #
  7. Tahir Ahmed wrote::

    I actually have a lot of respect for you Thibault and other Evangelists/engineers etc who are very well connected to the Flash community and are really approachable. This fact alone is so satisfying.

    Just wanted to bring it up and give credit where it is due; Big thanks.

    - Tahir.

    Friday, October 28, 2011 at 10:03 am #
  8. Bart wrote::

    Awesome, I was thinking about asking for this but once again you read my mind. Adobe must’ve realized that Capture Device support is one big advantage Flash has over HTML 5.

    And ppz: Please all be aware the Camera API is not limited to streaming from WebCam’s! It seems Flash uses all Capture Devices the OS has registered, for example my TV Tuner / Capture Card is listed, it’s really cool to mess with.

    Friday, October 28, 2011 at 11:07 am #
  9. Bart wrote::

    One thing: will Adobe be fixing that ugly and extremely buggy Privacy Panel piece of horror?

    So that people in all browsers can click on the Allow/Deny buttons and not be stuck with a blocked SWF?

    Even in WMODE opaque/transparent? Or when the SWF has odd-numbered dimensions? That would be realy bomb, then we can actually use the camera in the wild.

    Friday, October 28, 2011 at 11:12 am #
  10. Burak KALAYCI wrote::

    Hi Thibault,

    I’m a bit confused about SWF versions

    > swf version 14 is for FP11.1 and AIR
    3.1. Minor releases.

    This part is clear. SWF 14 is for FP11.1, not released yet.

    But then,

    > they will not be part of 11.2 and AIR 3.2, but will be part of a next release.

    So, these won’t be in FP 11.2.

    >just use -swf-version=15 to unlock them

    But they are in FP 11.2 *beta*, and version for them is 15.

    So my questions to clear my confusion:

    Is SWF 15 for F P11.2 or a newer “next release” version?

    Will these hidden gems be in release version of FP 11.2?

    Thanks, and congrats to you all.

    Best,
    Burak

    Friday, October 28, 2011 at 12:19 pm #
  11. Paul Neave wrote::

    This is awesome news! Thanks for listening, Thibault. I can’t wait to start adding this to my Webcam Toy app which creates camera effects with Pixel Bender.

    I guess we can create a fallback for this so it will work for users who do not yet have FP 11.2. Something like this:

    if (StageVideo["attachCamera"]) { StageVideo["attachCamera"](); }
    else { myVideo.attachCamera(); }

    That way we can compile against < FP 11.2 but still use these APIs if they are available. The same can be done for the VIDEO_EVENT event string.

    Paul.

    Friday, October 28, 2011 at 12:54 pm #
  12. jce wrote::

    Hi Thibault,
    Pleaaaaaaaase can we get some idea of when stageVideo for desktop air is going to be available?

    PLEASE!!

    Friday, October 28, 2011 at 1:06 pm #
  13. makc wrote::

    What object will fire frame events? Video? StageVideo? Camera?

    Friday, October 28, 2011 at 4:58 pm #
  14. Mike Haugland wrote::

    These changes are exactly what I need to help optimize a mobile project I’m working on. Are these API additions available for mobile as well?

    Friday, October 28, 2011 at 5:02 pm #
  15. Thibault Imbert wrote::

    jce,

    Let me follow up on this. I give you an update here very soon.

    makc,

    Camera will dispatch the videoFrame event.

    Mike,

    These APIs will be available for mobile too in the future, but not as the same time as on desktop.

    Thibault

    Friday, October 28, 2011 at 6:18 pm #
  16. Vasile wrote::

    any 3d drawing API (lines, circles, bezier curves) gpu powered?

    Friday, October 28, 2011 at 9:48 pm #
  17. Spencer wrote::

    With 3.2 I successfully loaded a 28330×3180 pixel jpg.
    Nice!!!!!!!!!

    Saturday, October 29, 2011 at 1:35 am #
  18. David Smith wrote::

    I agree with Tahir. The insights and information you bring forward here are greatly appreciated. Keep up the good work!

    Saturday, October 29, 2011 at 9:35 am #
  19. senthil kumaran wrote::

    Hi Spencer,

    I believe actually there is no limit for bitmap size.

    Saturday, October 29, 2011 at 1:32 pm #
  20. David Q Hogan wrote::

    Fantastic, the new frame event alone is going to help us a lot, thanks!

    What do you think of:

    function copyEncodedToByteArray( rect:Rectangle, destination:ByteArray ):Boolean;

    that would place a key or intermediate video frame encoded in the currently used video codec into a ByteArray, and return true for keyframe?

    Having similar functionality for the Microphone would be amazing, especially on mobile devices (our primary focus).

    Sunday, October 30, 2011 at 5:07 am #
  21. shachar oz wrote::

    i was nearly about to give up on Flash technology in order to create augmented reality applications and now you tell me about this.
    i have more hope to wait and see the results.
    :)

    Sunday, October 30, 2011 at 8:34 pm #
  22. Martin wrote::

    Hi Thibault,

    Please tell me whether it is possible use of Flash 11.x in adobe cs5 (flash prof. cs5 or flash builder 4 ) ? or I have to upgrade to cs 5.5 ?

    Please Answer …

    Sunday, October 30, 2011 at 9:23 pm #
  23. Thibault Imbert wrote::

    Hi Burak,

    Yes, SWF version for FP11.2/AIR3.2 is 15 but those features, even if they appear in these beta builds will not make it for 11.2 and 3.2.

    Hope this helps!

    Thibault

    Monday, October 31, 2011 at 6:26 am #
  24. Thibault Imbert wrote::

    Hi Martin,

    Yes you can target Flash 11 with Flash Builder 4, not sure for Flash Pro CS5, but with Flash Builder 4 for sure. Just use the right -swf-version as a compiler argument. For example, the Flash Player 11 APIs are -swf-version=13.

    Thibault

    Monday, October 31, 2011 at 7:02 am #
  25. Martin wrote::

    thank you Thibault .

    Monday, October 31, 2011 at 10:55 am #
  26. Tim wrote::

    Hi Thibault,

    Thank for the hidden gems. Is it possible to use color transform with StageVideo? Or is the best way to color transform the webcam stream by using Camera. drawToBitmapData ?

    Thanks Tim.

    Wednesday, November 2, 2011 at 5:26 pm #
  27. maouss wrote::

    Hi Thibault !
    Is there any comparation table of AIR 3 capabilities for different profiles ? as for AIR 2.0 ( http://help.adobe.com/en_US/air/build/WS144092a96ffef7cc16ddeea2126bb46b82f-8000.html ) ?
    Actualy, I’d like to know if DatagramSocket will be available for AIRforTV 3 :)
    Could you also tell me if stage3D layer transparency will be enabled in the future ? I work on samsung smartTV platform and this transparency is very important to provide good accelerated interface over video stream.
    Thanks a lot by advance for your answers
    morgan

    Tuesday, November 8, 2011 at 12:44 pm #
  28. Daniel Hai wrote::

    How about:

    Camera.drawToTexture(texture:Texture)

    :)

    Wednesday, November 9, 2011 at 11:19 pm #
  29. jce wrote::

    Hi Thibault, any update yet on StageVideo for desktop air??

    Wednesday, November 23, 2011 at 8:56 pm #
  30. What the hell are you guys doing? I don’t care if you give me better camera support with lower CPU. Why the hell did you get rid of Alchemy, and in a point release at that! WTF are you guys doing? What am I meant to use now?

    Thursday, November 24, 2011 at 12:52 am #
  31. Ventoline wrote::

    Hi there,

    Looks like an amazing improvement! Would there be an example written for Stagevideo.attachcamera(), I cant seem to get my head around it! ^^

    Thursday, November 24, 2011 at 8:52 pm #
  32. paha wrote::

    @Tarwin

    Alchemy 2 I suppose? As told in other thread, the original Alchemy code will work when targeting swf 13 or lower.

    Friday, November 25, 2011 at 8:33 pm #
  33. @paha

    Which thread actually says this? The official blog says that 11.2 will break this support (saying that means SWF v13)

    http://blogs.adobe.com/flashplayer/2011/09/updates-from-the-lab.html

    So if you want speed you just have to hope that your user will not upgrade, and you have to create two versions, one for the old, and one for the new users.

    I still have not seen any good reason why they are not just stopping development,but also breaking it. That simple courtesy would be the least they could do!

    As you can see from the comments at this blog though it doesn’t seem like anyone reads it – or there would have been a big fuss a lot earlier.

    Monday, November 28, 2011 at 2:54 pm #
  34. Thibault Imbert wrote::

    Hi Tahir and David,

    Thanks for the kind words.

    Thibault

    Thursday, December 29, 2011 at 10:09 am #
  35. Thibault Imbert wrote::

    Hi Tarwin,

    Yes Alchemy 2 would be the path forward. But I know that being in the dark for this period of time can be annoying.

    Keep in mind that Alchemy has always been an experiment project, and even if it became an habit to use it for some stuff, we were always clear about its “experiment-ness” and the possibility that it would not be supported for ever.

    Now, you could tell me, well, then give me a better compiler and I would not need Alchemy cause I do not need to cross compile my code, I want better speed with AS3 :)

    And we are working on that :)

    Thibault

    Thursday, December 29, 2011 at 10:18 am #
  36. Matt wrote::

    Hi Thibault,
    can you tell me if the copyToBitmapdata and video_frame event is/will also be available to normal videos/netstreams?

    Thanks,
    Matt

    Wednesday, January 11, 2012 at 2:30 pm #
  37. Tim wrote::

    Hi all,

    Are the hidden gems not working anymore in the FP 11.2 beta 3 release?

    Saturday, January 14, 2012 at 5:23 pm #
  38. Thibault Imbert wrote::

    Hi Tim,

    Yes, not anymore. They will be back in a next release, very likely 11.3.

    Thibault

    Sunday, January 15, 2012 at 12:03 pm #
  39. Hi Thibault,

    So if Adobe disables Alchemy 1 opcodes in Flash Player 11.2 or 11.3, then doesn’t that mean that after the player release games could not use Alchemy 1 opcodes and Starling at the same time?

    If I’m understanding correctly, Starling uses Flash Player 11 features, but if we want want to use Alchemy 1 features, we would have to target Player 10 in order to keep it future proof?

    Would this mean that games that use both Haxe and Starling would stop working in future versions of Flash Player?

    Lee Brimelow just released a tutorial using Nape (which uses Haxe) and Starling:
    http://www.leebrimelow.com/?p=3245
    Would this stop working in future versions of Flash player?

    Thanks in advance for your clarifications!

    Jacob

    Saturday, January 21, 2012 at 9:38 am #
  40. gepatto wrote::

    Can anyone tell me what is needed to compile to the 3.2 namespace. I downloaded the correct playerglobal.swc, and changed the application descriptor to 3.2 and added -swf-version=15 (using flex4.6 sdk, and also tried an older 4.5sdk with air3.1 overlayed), But i still get :invalid application descriptor: Unknown namespace: http://ns.adobe.com/air/application/3.2

    or do I have to wait for a public Air3.2 sdk?

    best regards

    Monday, January 23, 2012 at 2:08 pm #
  41. mrballistic wrote::

    Man… these can’t come soon enough for me. i’m writing a camera app that builds histograms on the fly for ios, and it’s wicked slow without access to stagevideo and the other improvements above.

    please please please release the sdk and packager for air 3.2 (even in beta) rsn!

    Wednesday, January 25, 2012 at 1:06 am #
  42. mrballistic wrote::

    gepatto — you’ll need to wait for the sdk

    Wednesday, January 25, 2012 at 1:07 am #
  43. Eric Hélier wrote::

    I’d love to get a clear answer to Jacob’s questions too.
    We are in the process of building a sdk for augmented reality applications, and targeting Flash without Stage3D AND Alchemy support is almost impossible for us…

    Joa Ebert stated on another blog that even loading a “swf-version 12″ alchemy swf inside a “swf-version 13″ standard application won’t work. Is that true ?
    This could be a way for existing libraries based on alchemy to continue working.

    Saturday, March 3, 2012 at 12:34 am #
  44. Matt Lockyer wrote::

    Hello!

    Now that the RC SDK is out, how can we use these little gems?

    I have tried to use the above methods but with no success.

    Yes I have layered the SDK over my previous install…

    Thursday, March 8, 2012 at 11:38 pm #
  45. Hello what is the name of AIR SDK release that has these gems now? Can we still find it on adobes website?

    Thursday, March 15, 2012 at 5:22 pm #
  46. Maxxus wrote::

    Hi!
    Flash 11.2 has been released, but such stuff is not there! When/how we can use that methods?

    Monday, April 2, 2012 at 12:46 am #
  47. Ben Ursu wrote::

    Please bring StageVideo.attachCamera back. StageVideo.drawToBitmapData would also be awesome. Both unique and very special hardware efficiency to software functionality features of mobile devices. Great work!

    Sunday, April 29, 2012 at 9:08 am #
  48. David Q Hogan wrote::

    Hi Thibault,

    Do you know if the videoFrame event and Camera pixel functions will make it into Air 3.3? As far as I can tell they’re not in Air 3.3 beta 2. I’m currently using an ‘on enter frame’ and bitmap comparison hack which doesn’t run so well on the iPhone 3GS :)

    Cheers,
    Dave

    Monday, April 30, 2012 at 1:26 am #
  49. This sounds like exactly what I need for my project. Unfortunately these things do not appear to be in 3.2 for iOS. Any expectation that they will be some day?

    Tuesday, June 5, 2012 at 3:50 am #
  50. Marc Tanenbaum wrote::

    Thibault,

    So here we are and Air 3.3 has shipped, but (at least) stageVideo.attachCamera() still isn’t available? Is that correct? Will it be released any time soon?

    Thanks for all your hard work.

    Wednesday, June 20, 2012 at 6:02 pm #
  51. Thibault Imbert wrote::

    Hi Marc,

    Yes, all these will be available in FP11.4/AIR 3.4 coming soon :)

    Thibault

    Tuesday, June 26, 2012 at 7:26 am #
  52. pina colada wrote::

    Flash évolue vite (et bien) depuis quelques mois, particulièrement. La concurrence, sans doute ? ;-)
    Merci en tous cas à l’équipe qui lui donne des ailes (le son de synthèse, la 3D native, le bouton de droite, le retour du release outside m’ont presque fait pleurer de joie !)
    Et, si on peut être gourmand, j’aurais une requête (toute simple, me semble-t-il): la possibilité de zipper et de dézipper SANS LIBRAIRIE tierce, afin d’accéder aux fichiers archives avec le bytearray (déjà capable de compresser)
    You can do it !

    Sunday, July 1, 2012 at 7:31 pm #

Trackbacks/Pingbacks (2)

  1. [...] guys remember the post I did a little while ago entitled Flash Player 11.2/AIR 3.2 beta hidden gems? I mentioned at that time camera support for StageVideo, here it [...]

     
  2. [...] guys remember the post I did a little while ago entitled Flash Player 11.2/AIR 3.2 beta hidden gems? I mentioned at that time camera support for StageVideo, here it [...]