Recording video from opengles

gaurav.singh

Hello friends,

I am using Sparrow framework v2.0 in my app to display a complex animation. I want to create the video for this animation which is displayed in SPViewController.
By taking inspiration from the category class SPDisplayObject+Image at
http://wiki.sparrow-framework.org/users/shilo/extensions/spdisplayobject_image
I have created a class that creates a video. This class writes a frame to the video when SPViewController's -glkView:drawInRect is called.

But the class is only working in ios simulator and crashes in iPhone/iPad whenever glreadpixels() is encountered.

Here is my code:

SPViewController

- (void)glkView:(GLKView *)view drawInRect:(CGRect)rect
{
    @autoreleasepool
    {
        if (!_root)
        {
            [self readjustStageSize];
            [self createRoot];
        }
        [Sparrow setCurrentController:self];
        [EAGLContext setCurrentContext:_context];
        glDisable(GL_CULL_FACE);
        glDisable(GL_DEPTH_TEST);
        glEnable(GL_BLEND);
        [_support nextFrame];
        [_stage render:_support];
        [_support finishQuadBatch];
        if (_statsDisplay)
            _statsDisplay.numDrawCalls = _support.numDrawCalls - 2; // stats display requires 2 itself
        #if DEBUG
        [SPRenderSupport checkForOpenGLError];
        #endif
        if ([self.movieWriter isRecording]) {
            [self.root writePixelBufferToMovieWriter];
        }
    }
}

SPDisplayObject+movieWriter

- (void)writePixelBufferToMovieWriter{
    SPViewController *sparrowVC = Sparrow.currentController;
    int contentScaleFactor = Sparrow.contentScaleFactor;
    int width = self.width*contentScaleFactor;
    int height = self.height*contentScaleFactor;
    int bufferLength = width*height*4;
    unsigned char buffer[bufferLength];
    //[SPRenderSupport clearWithColor:0x0 alpha:1.0f];
    //[self render:viewController.renderSupport];
    glReadPixels(0, (Sparrow.stage.height*contentScaleFactor)-height, width, height, GL_BGRA, GL_UNSIGNED_BYTE, &buffer);
    CVPixelBufferRef pixel_buffer = NULL;
    CVPixelBufferCreateWithBytes(NULL, width, height, kCVPixelFormatType_32BGRA, buffer, 4 * width, NULL, 0, NULL, &pixel_buffer);
    [sparrowVC.movieWriter writeToVideoUsing:pixel_buffer];

    CVPixelBufferRelease(pixel_buffer);
}

What I am doing wrong?
By the way I am pretty new to openGLES.

Any help will be appreciated. Thanks

gaurav.singh

forgot to include tags.

Shilo

A crash is usually due to exc_bad_access, which usually means that you are trying to access an invalid or released object. I have had memory problems where I would get a crash on the device, but not on the simulator. You can track down the cause of it more easily if you enable NSZombies. Can you verify what the cause of the crash is?

Have you tried moving your code to other locations in the draw method? Such as right after the "render" call and right before the "finishQuadBatch" call?

Also, what part of the screen are you trying to capture? Are you trying to capture: the whole stage, a specific bounds in the stage, or a specific display object and no other objects?

gaurav.singh

Thanks Shilo for this early reply.

I already have NSZombie enabled.

I have added SPViewController's view as subview to my required view controller because I need to display it in a small area.
I am using a SPSprite as the root of SPViewController. I want to record all that is happening in the root SPSprite.

I haven't yet tried to move the code in the draw method but I am just going to try that. I'll report back after doing that.

gaurav.singh

I have tried moving the code to multiple positions in the draw method but still getting same crash "EXC_BAD_ACCESS" in device.

I am unable to pass the line containing glreadpixels().
Also, in the category class SPDisplayObject+Image , I am getting the same crash while creating a snapshot image.

I am unable to find the cause of the crash.

Shilo

I am using a SPSprite as the root of SPViewController. I want to record all that is happening in the root SPSprite.

Does that mean you will be recording everything on the stage? If so, I suggest you just use GLKView's "snapshot" method. It does all the work for reading the pixels and creating the UIImage.

gaurav.singh

GLKView's snapshot method returns a UIImage which is good for screenshot but I guess is not suitable for writing a movie. Also snapshot method cannot be called from -glkView:drawInRect method hence another timer will be needed.

When tested in simulator I find the resultant image from SPDisplayObject+Image class is far better than the one from GLKView's snapshot. In GLKView's snapshot image, a white boundary is visible around the images(SPImage) added to root Sprite in opengl which is not visible in simulator.

Shilo

Oh, I didn't realized that you weren't creating UIImage. That makes sense to skip that overhead. Those are some great finds!

I wonder how you have had some success with my SPDisplayObject+Image category. I recently tried it again and wasn't able to get it to work, it seems there must be some kind of "condition" for it to properly read the pixels of the specific display object. I also am not sure if I am properly rendering the object on demand.

gaurav.singh

@Shilo! Is there any solution to my problem that you find or suggest?