BitmapData API’s additions

Flash Player Fur
One of the most exciting API in the Flash Player is definitely the BitmapData API. I remember, its introduction in Flash Player 8 (Maelstrom), so many things suddenly became possible thanks to it.

Yesterday with Tinic, we were having a drink and talking about this nice API in the Flash Player. We were wondering which API's could be added that would help you guys to make your every day work with the BitmapData API easier.

I am sure you may need some simple additions but also some more complex additions for your bitmap based frameworks (AR, games, etc.). We thought about some stuff but I do not want to influence you guys and I would be happy hearing your wishes.

I went through the Jira and looked all the requests regarding the BitmapData API and I found some great requests. Mario, definitely raised its voice there, I confirm ;)

If you have never been there to log it, you can raise your voice here. If you already logged it, just post a link to it in case I missed it.

Thank you Nicolas for the online fur app ;)

Comments (43)

  1. Awesome, I will go on tomorrow night and post my thoughts after going through Jira! Thanks Thibault.

    Sunday, August 1, 2010 at 3:21 am #
  2. xethorn wrote:

    One thing would be great: is to allow the prototype to exist in every flash object and not just some of them.

    @xethorn

    Sunday, August 1, 2010 at 3:44 am #
  3. someone wrote:

    not really BitmapData related, but how about not receiving mouse events with transparent parts of Bitmap?

    Sunday, August 1, 2010 at 10:01 am #
  4. Jerome wrote:

    Hello Tinic & Tibo :)

    There are two very basic things that can be greatly improved:
    - resize
    - sharpen

    You probably know that the current bitmapdata resize (bilinear) is not perfect.
    If you look this post http://tinyurl.com/cmw2rx (sorry in french)

    If you look Hero (Flex 4.5). They want to introduce a new BitmapImage, with a new smoothQuality option to resolve the BitmapData resize problem. See the picture here: http://tinyurl.com/33vtrev

    Know if you look imagemagik, it has several options to resize an image. http://tinyurl.com/29rujjw
    So you can choose if you want speed/low quality or medium speed/middle quality or low speed/high quality. Exactly like in Flash filters (low, medium, high)

    Please take a look at the different resize algorithm, it’s perfect explanation : http://tinyurl.com/28epsaj

    //

    There is also a great lack of filter to improve the sharpness of an image.
    Everyone know the unsharp mask filter in Photoshop http://tinyurl.com/bw5yt
    It’s not perfect, but it’s fast, and have 3 params to adjust it http://tinyurl.com/3285dlm

    Currently in Flash you can do it with a Convolution Filter but the result is bad and there is no possibility to fine tune the parameters.

    If you want use Flash online or in an AIR application that work with images you probably want a better quality that we have today. The current implementation is perfect for speed (website rendering) but not at all for highest quality (RIA rendering)

    Thanks for reading, have a good drink!

    Sunday, August 1, 2010 at 10:27 am #
  5. Philippe wrote:

    Simple: look at HTML5 Canvas methods.

    Sunday, August 1, 2010 at 11:12 am #
  6. I’d like to see 3 new native filters:
    Median, Minimum, Maximum – those are algorithms where Pixel Bender due to it’s per pixel processing can only suck but which can be written very optimized in native code. All of these filters should have the option to either check a square area or a circular one around each pixel.

    A version of the threshold filter that allows to set a range for each channel (right now you can only check if a pixel is below or above the threshold). Also an option to set a color” for the “false” case:
    threshold2(map:BitmapData,rect:Rectangle,point:Point,minValue:uint,maxValue:uint,mask:uint,color1:uint,color2:uint,useColor2:Boolean,copyPixels:Boolean);

    As I already posted on jira:
    A bitmapData.fill( map:BitmapData, matrix:Matrix ) method that works just like graphics.beginBitmapDataFill() in order to avoid having to create a shape first and then draw() if one wants to create repetitive patterns.

    A tolerance setting for floodFill() like Photoshop’s magic wand. Also flood fill should return a vector or all pixel indices that have been filled. This can be very helpful for all kinds of tracking or tracing applications.

    A property that allows to switch off pre-multiplied alpha in a bitmapdata. This would be only useful for people who use bitmaps for calculations and it would break the compositing, so it should come with a warning.

    A way to change the dimensions of a bitmapdata without destroying the reference to it: bitmapData.setNewSize( align:String, backgroundColor:uint);

    A plasma fractal filter.

    A new floating point BitmapData like the one pixel bender uses internally. Would allow for more precision and ideally reduce the amount of conversion that has to go on for pixel bender processing.

    Some more obscure ideas:
    flipRotate() – would allow to rotate the bitmap in 90° steps and/or mirror it. This will actually change the dimensions of the bitmap if width and height are different. I’m well aware that this can already be done with draw() and a new BitmapData instance.

    Okay, enough for now ;-)

    Sunday, August 1, 2010 at 12:43 pm #
  7. Jerome wrote:

    I think Thibault, Tinic and Mario should have a drink together more often! ;)

    Sunday, August 1, 2010 at 1:33 pm #
  8. Joa Ebert wrote:

    bitmapData.draw() with an optional BitmapData as an alpha mask would be nice.

    Sunday, August 1, 2010 at 2:28 pm #
  9. ptitJean wrote:

    Yep,

    a getBitmapData() on each displayObject that return the bitmapData after a cacheAsBitmap to avoid all the draw() process on a single displayObject.
    Or
    setCacheAsBitmap( boolean ) : BitmapData
    return null if set to false(no cache)

    Voilà :-)

    Sunday, August 1, 2010 at 2:28 pm #
  10. ciao Thimbault
    here’s my raised voice :)

    create a bitmapdata with a specified amount of channels (mono channel is happening to me quite frequently)

    determination of the color depth per channel (not always a byte)

    applyShaderFilter(pixelbenderfilter,params)

    a better implementation of perlinNoise (mario did one some time ago)

    further, a plus to mario’s bitmapData.fill(map:BitmapData, matrix:Matrix)

    looking forward to meet you at fotb :)
    ciao

    Sunday, August 1, 2010 at 3:27 pm #
  11. Rezmason wrote:

    Man, if you’d implement Mario’s idea for a floating point BitmapData variety, I would FLIP OUT. We’d be seeing third party ActionScript libraries popping up for things like tracking features in a video buffer and optical flow algorithms.

    Sunday, August 1, 2010 at 3:31 pm #
  12. Rezmason wrote:

    Most methods on BitmapData, as well as Graphics.beginBitmapFill() and GraphicsBitmapFill, allow us to specify an affine transformation matrix to apply to the image, so we can mess with it. I think these functions should also allow us to specify a ColorTransform to be applied; if I have a handful of graphics paths with bitmap fills, and I want each of them to be tinted different colors, I shouldn’t have to draw the paths to separate Shapes’ Graphics objects.

    Sunday, August 1, 2010 at 3:39 pm #
  13. foo wrote:

    let’s try again, shall we. I want same kind of params in beginBitmapFill that you have in draw.

    Sunday, August 1, 2010 at 5:59 pm #
  14. ptitJean wrote:

    Hello!!!

    A getBitmapData() on each DisplayObject to get a bitmapData after a cacheAsBitmap to avoid draw() process on a single DisplayObject. It will return null if cacheAsBitmap is set to false.
    Or something like that to replace cacheAsBitmap property:
    setCacheAsBitmap(Boolean) : BitmapData
    it will return null if set to false, if not it will return the BitmapData

    Voilà :-)

    Sunday, August 1, 2010 at 6:33 pm #
  15. Thibault Imbert wrote:

    Thanks guys, very valuable stuff.

    Philippe,

    Yes, I have done that. Some good news soon ;)

    Thibault

    Sunday, August 1, 2010 at 8:28 pm #
  16. Just so it does not get lost in Jira: an option use the draw() command with QUALITY_LOW (non-antialiased) without having to change the global stage quality. There are often cases where no stage is available to the object which requires to pass in a stage from outside the class to that does the draw() or as in AIR it is entirely impossible to set the quality to low.

    Monday, August 2, 2010 at 11:51 am #
  17. Ah, one more – a possibility to apply multiple filters in a row on the same BitmapData object so that there is no internal copying back and forth of temporary bitmaps or marshalling taking place.

    Monday, August 2, 2010 at 11:56 am #
  18. Three more:
    - a “falloff” parameter for Perlin Noise
    - a Worley Noise generator
    - a Simplex Noise (aka Perlin Noise 2.0) with the ability to control at least the x,y and z offsets

    Monday, August 2, 2010 at 12:11 pm #
  19. Hi,

    Maybe that the fill method return the filled region as a Rectangle. I think it will be much faster than doing it in two separate pass.

    I’m needing more speed in the new color barcode reader i’m developing at my job.

    Monday, August 2, 2010 at 3:25 pm #
  20. Jake Lewis wrote:

    An additional, optional, parameter of sourceRect:Rectangle for the getColorBoundsRect() method would speed up blob detection.

    Monday, August 2, 2010 at 4:25 pm #
  21. Rezmason wrote:

    I want a marioKlingemann() method on BitmapData that’d return a 140 character Unicode String representing your image as a set of colored regions defined by a Voronoi diagram.

    It should also trace out “Thank you for spelling my name correctly.”

    Monday, August 2, 2010 at 5:34 pm #
  22. Benny wrote:

    I would like to see an option to export the bitmap data as JPEG or PNG data. It would also be great to have the ability to resize by seam carving.

    Monday, August 2, 2010 at 9:38 pm #
  23. Thibault Imbert wrote:

    Thanks guys. Just to let you know, we had another drink and we are going to work on that. I cannot commit yet, but looks promising. Stay tuned :)

    Thibault

    Tuesday, August 3, 2010 at 6:38 am #
  24. Mark wrote:

    It would be great to have an new floodfill like option with some feather-settings (like PS) and also to to do a global floodfill (not starting from a point, but over all colors inside the bitmap, so it would be more easy to do chroma keying.

    Tuesday, August 3, 2010 at 1:35 pm #
  25. Mahron wrote:

    An isTransparent property to know if the bitmapdata has transparent pixels.
    And as benny said jpg and png encoder. This is really needed to send pics as bytearrays to the server. Encoding in as3 in too slow for large images.

    Tuesday, August 3, 2010 at 5:25 pm #
  26. Mahron wrote:

    Native QR code generator and reader would be nice to. With flash coming to mobiles it would be relevant now that the camera can be pointed towards anything.

    Tuesday, August 3, 2010 at 5:41 pm #
  27. - Definetly a Bicubic interpolation option
    - Why not the replica of Graphics class methods as lineTo without the need of creating a vector shape and the draw() it.
    - A plus to the Piergiorgio request of N channels images (mono-channel image could save some precious ram)
    - Some coolest noise function
    - .resize method with interpolation option would be very cool limiting the need of different instances.

    However bitmapData class rox! Best invention since flash itself :)

    Wednesday, August 4, 2010 at 9:38 am #
  28. re #24: Mark, you can do the global floodFill as you intend it by using threshold. But for Chroma keying I’d rather recommend to use Pixel Bender.

    Wednesday, August 4, 2010 at 12:16 pm #
  29. Joa Ebert wrote:

    Some way to determine which DisplayObject is responsible for a SecurityError when using BitmapData.draw would be great as well.

    Or a way to say that you want to ignore those DisplayObjects when drawing.

    Wednesday, August 4, 2010 at 12:21 pm #
  30. it would be nice to have a luminance map blendmode (black and white) that works like the alpha blendmode – i know its possible to fix with color matrix but thats a hastle –

    also i thought of a cool filter that works like the displacementMapFilter but instead of moving the pixels it applies blur in the direction of the pixels with the force of the colors like in the displacementmap filter

    Wednesday, August 4, 2010 at 12:33 pm #
  31. and also a chromatic aboration filter

    Wednesday, August 4, 2010 at 12:36 pm #
  32. Daniel Hai wrote:

    - MipMapping – be able to turn it on or off (memory footprint on large bitmaps is too much, and making an odd numbered bitmap isn’t desired either)
    - Increase Pixel Bender speed — getting great framerates on larger bitmaps is very difficult – sometimes doing a simple translation to the same color can take 8-9 ms on a 480×360 bitmap.
    - A way to link vectors/bytearrays/bitmaps. setPixels and getPixels are very very expensive, if they could somehow be attached without having to create new vectors/bytearrays/bitmaps that would be great. Try writing something that saves to disk — it’s impossible to get framerates that are useful, since getPixels or getVector takes a minimum of 8ms on 320×240 image.
    - A better way to deal with alpha masking – right now I am drawing shapes, blurring them, and then having to create a new bitmap instance and then copy the new data back in.

    Wednesday, August 4, 2010 at 6:55 pm #
  33. Oh now I remembered an important one: really big bitmaps are currently a problem memory-wise since they need a contiguous area of free memory to be allocated. The more memory gets fragmented during the running time of an app the higher is the probability that it is not possible to allocate the memory required for these big bitmaps whilst at the same time it still might be possible to allocate space for 4 bitmaps that are each a quarter of the size of that map. So ideally Flash should handle (big) bitmaps internally in tiles the way Photoshop does it. Of course doing that would require a lot of changes to the way filters work internally, at least those that access pixels outside the tile they are on, so I fear we won’t get that feature.

    Thursday, August 5, 2010 at 5:09 pm #
  34. Adrian Parker wrote:

    Please please please put in some methods that make loading from / saving to file and resizing more easy.

    Thursday, August 5, 2010 at 8:34 pm #
  35. Daniel Hai wrote:

    Another big one for me: bitmaps should be able to refer to a segment of a bitmap.

    bitmap.sourceRect = some segment;

    Thursday, August 5, 2010 at 11:27 pm #
  36. Gene Beidl wrote:

    I would LOVE to have transparent parts of bitmaps (alpha = 0) pass mouse events to objects that are visible behind the transparent portions of the bitmap. Currently, if I have a bitmap that is completely transparent, I can see lots objects behind it, but can’s select anything.

    I would also love this ability for alpha-enabled video!!

    Sunday, August 8, 2010 at 2:19 am #
  37. Jerod Moore wrote:

    I would LOVE to see better commenting, on a more human level, in the API.

    Monday, August 9, 2010 at 1:26 pm #
  38. Edvin wrote:

    I see someone allready posted about a float type bitmap, i can only second this, having colors in 0-1 range would make life much easier.

    Another thing I would love to have is a better way to manipulate pixels, or faster atleast. Some way of iterating through a bitmapdata in a fast, low-overhead, manner would be a great asset to the flash community.

    A not so high priority but would be great is a base class for pixel data, and maybe the ability to extend it and pass in our own pixel type to the bitmapdata object where we could override the final value getter and compute our own. I know this is pretty much pixelbender territory, but would still be a nice feature.

    Monday, August 9, 2010 at 2:04 pm #
  39. travikk wrote:

    I’ve posted some of my ideas on my blog. You can find it here:
    http://flexythoughts.org/2010/08/03/bitmapdatautility-painting-book-in-as3/

    Tuesday, August 10, 2010 at 11:48 pm #
  40. nicoptere wrote:

    (after the war…)
    if I ruled the world:
    - a colortransform &| matrix in the copyPixels method (I think it’s not possible otherwise it would already be there).
    - embed the JPGEncode, PNGEncode, (GIF?!) + add a save method for lazy bastards like me (+a load as said in #34).
    - http://www.bytearray.org/?paged=14 definitely
    - implement the main binary morphomaths operators: erosion, dilation, opening, closing (+ the ability to set a bitmapdata as the structuring element)
    -fix the bug when a bitmap’s alpha is greater than 1 (ok, it should not be greater than 1 …)

    +1 to #4, #6, #10, #17, #18, #32
    - #29 or force the implmentation of IBitmapdrawable on all the
    potentially displayable objects ^^

    more generic to all displayObjects ; implement the photoshop blendmodes especially: HUE, COLOR, SATURATION & LUMNIOSITY as said in #30

    the BitmapData class is already my favorite class so no big deal if all this is not implemented soon,
    thanks for the trackback :)

    Tuesday, August 17, 2010 at 12:16 pm #
  41. Nico wrote:

    “more generic to all displayObjects ; implement the photoshop blendmodes especially: HUE, COLOR, SATURATION & LUMNIOSITY as said in #30″

    What about those in:
    flex/sdk/trunk/frameworks/projects/framework/src/mx/graphics/shaderCl asses/

    Sunday, September 12, 2010 at 1:42 am #
  42. Alan Shaw wrote:

    Obviously I should be tracking this blog more often… May I LOUDLY vote for floating-point and for optional turning off of premultiplied alpha.

    Saturday, October 2, 2010 at 7:19 am #
  43. Lukasz Kmiecik wrote:

    I would like to have additional parameter (or maybe inside beginBitmapFill method) from 0.0 to 1.0 – multiplier of RGB values.
    It will be usable for light calculations when triangle is half visible you can set bitmapData multiplier to 0.5 and all other API works on half visible bitmap (not alpha but RGB). What is important that mentioned multiplier should not change bitmap pixels inside bitmapData!

    Monday, October 11, 2010 at 12:54 pm #

Trackback/Pingback (1)

  1. BitmapData API’s additions | Lively Flash Tuts on Sunday, August 1, 2010 at 3:50 pm

    [...] Direct Link [...]