代码之家  ›  专栏  ›  技术社区  ›  Perraco

在Android camera2下将YUV_420_888转换为位图的图像不正确

  •  5
  • Perraco  · 技术社区  · 7 年前

    我试图将YUV_420_888图像转换为位图,来自camera2预览版。但输出图像的颜色不正确。

    接下来是Im运行以生成位图的测试代码。只是测试代码,所以请不要做任何代码审查无关的因素,如位图正在回收,或渲染脚本是不断创建。这段代码只是测试从YUV到RGB的转换,仅此而已。

    由于其他因素,代码是从API 22及以上版本运行的,因此使用特定于RenderScript的 ScriptIntrinsicYuvToRGB 应该足够了,而不必使用旧的手动转换,因为缺乏适当的YUV\u 420\u 888支持,只有在以前的Android版本中才需要。

    由于RenderScript已经提供了一个专用的ScriptIntrinsicYuvToRGB,用于处理所有类型的YUV转换,我认为问题可能在于如何从Image对象获取YUV字节数据,但我不知道问题出在哪里。

    要在Android Studio中查看输出位图,请在位图中放置断点。recycle(),因此在它被回收之前,您可以在变量调试窗口中使用–view bitmap–选项查看它。

    如果有人能发现转换有什么问题,请告诉我:

    @Override
    public void onImageAvailable(ImageReader reader)
    {
        RenderScript rs = RenderScript.create(this.mContext);
    
        final Image image = reader.acquireLatestImage();
    
        final Image.Plane[] planes = image.getPlanes();
        final ByteBuffer planeY = planes[0].getBuffer();
        final ByteBuffer planeU = planes[1].getBuffer();
        final ByteBuffer planeV = planes[2].getBuffer();
    
        // Get the YUV planes data
    
        final int Yb = planeY.rewind().remaining();
        final int Ub = planeU.rewind().remaining();
        final int Vb = planeV.rewind().remaining();
    
        final ByteBuffer yuvData = ByteBuffer.allocateDirect(Yb + Ub + Vb);
    
        planeY.get(yuvData.array(), 0, Yb);
        planeU.get(yuvData.array(), Yb, Vb);
        planeV.get(yuvData.array(), Yb + Vb, Ub);
    
        // Initialize Renderscript
    
        Type.Builder yuvType = new Type.Builder(rs, Element.YUV(rs))
                .setX(image.getWidth())
                .setY(image.getHeight())
                .setYuvFormat(ImageFormat.YUV_420_888);
    
        final Type.Builder rgbaType = new Type.Builder(rs, Element.RGBA_8888(rs))
                .setX(image.getWidth())
                .setY(image.getHeight());
    
        Allocation yuvAllocation = Allocation.createTyped(rs, yuvType.create(), Allocation.USAGE_SCRIPT);
        Allocation rgbAllocation = Allocation.createTyped(rs, rgbaType.create(), Allocation.USAGE_SCRIPT);
    
        // Convert
    
        yuvAllocation.copyFromUnchecked(yuvData.array());
    
        ScriptIntrinsicYuvToRGB scriptYuvToRgb = ScriptIntrinsicYuvToRGB.create(rs, Element.YUV(rs));
        scriptYuvToRgb.setInput(yuvAllocation);
        scriptYuvToRgb.forEach(rgbAllocation);
    
        // Get the bitmap
    
        Bitmap bitmap = Bitmap.createBitmap(image.getWidth(), image.getHeight(), Bitmap.Config.ARGB_8888);
        rgbAllocation.copyTo(bitmap);
    
        // Release
    
        bitmap.recycle();
    
        yuvAllocation.destroy();
        rgbAllocation.destroy();
        rs.destroy();
    
        image.close();
    }
    
    3 回复  |  直到 7 年前
        1
  •  8
  •   Perraco    5 年前

    回答我自己的问题,实际的问题是,我怀疑我是如何将图像平面转换成ByteBuffer的。接下来是解决方案,它应该适用于NV21和;YV12。由于YUV数据已经在不同的平面中出现,因此只需要根据行和像素步长以正确的方式获取数据。还需要对数据传递到RenderScript内部的方式进行一些小的修改。

    注意:对于生产优化的onImageAvailable()不间断流,在进行转换之前,应将图像字节数据复制到单独的缓冲区,并在单独的线程中执行转换(取决于您的要求)。但由于这不是问题的一部分,在下一个代码中,转换直接放在onImageAvailable()中,以简化答案。如果任何人需要知道如何复制图像数据,请创建一个新的问题,让我知道,这样我将分享我的代码。

    @Override
    public void onImageAvailable(ImageReader reader)
    {
        // Get the YUV data
    
        final Image image = reader.acquireLatestImage();
        final ByteBuffer yuvBytes = this.imageToByteBuffer(image);
    
        // Convert YUV to RGB
    
        final RenderScript rs = RenderScript.create(this.mContext);
    
        final Bitmap        bitmap     = Bitmap.createBitmap(image.getWidth(), image.getHeight(), Bitmap.Config.ARGB_8888);
        final Allocation allocationRgb = Allocation.createFromBitmap(rs, bitmap);
    
        final Allocation allocationYuv = Allocation.createSized(rs, Element.U8(rs), yuvBytes.array().length);
        allocationYuv.copyFrom(yuvBytes.array());
    
        ScriptIntrinsicYuvToRGB scriptYuvToRgb = ScriptIntrinsicYuvToRGB.create(rs, Element.U8_4(rs));
        scriptYuvToRgb.setInput(allocationYuv);
        scriptYuvToRgb.forEach(allocationRgb);
    
        allocationRgb.copyTo(bitmap);
    
        // Release
    
        bitmap.recycle();
    
        allocationYuv.destroy();
        allocationRgb.destroy();
        rs.destroy();
    
        image.close();
    }
    
    private ByteBuffer imageToByteBuffer(final Image image)
    {
        final Rect crop   = image.getCropRect();
        final int  width  = crop.width();
        final int  height = crop.height();
    
        final Image.Plane[] planes     = image.getPlanes();
        final byte[]        rowData    = new byte[planes[0].getRowStride()];
        final int           bufferSize = width * height * ImageFormat.getBitsPerPixel(ImageFormat.YUV_420_888) / 8;
        final ByteBuffer    output     = ByteBuffer.allocateDirect(bufferSize);
    
        int channelOffset = 0;
        int outputStride = 0;
    
        for (int planeIndex = 0; planeIndex < 3; planeIndex++)
        {
            if (planeIndex == 0)
            {
                channelOffset = 0;
                outputStride = 1;
            }
            else if (planeIndex == 1)
            {
                channelOffset = width * height + 1;
                outputStride = 2;
            }
            else if (planeIndex == 2)
            {
                channelOffset = width * height;
                outputStride = 2;
            }
    
            final ByteBuffer buffer      = planes[planeIndex].getBuffer();
            final int        rowStride   = planes[planeIndex].getRowStride();
            final int        pixelStride = planes[planeIndex].getPixelStride();
    
            final int shift         = (planeIndex == 0) ? 0 : 1;
            final int widthShifted  = width >> shift;
            final int heightShifted = height >> shift;
    
            buffer.position(rowStride * (crop.top >> shift) + pixelStride * (crop.left >> shift));
    
            for (int row = 0; row < heightShifted; row++)
            {
                final int length;
    
                if (pixelStride == 1 && outputStride == 1)
                {
                    length = widthShifted;
                    buffer.get(output.array(), channelOffset, length);
                    channelOffset += length;
                }
                else
                {
                    length = (widthShifted - 1) * pixelStride + 1;
                    buffer.get(rowData, 0, length);
    
                    for (int col = 0; col < widthShifted; col++)
                    {
                        output.array()[channelOffset] = rowData[col * pixelStride];
                        channelOffset += outputStride;
                    }
                }
    
                if (row < heightShifted - 1)
                {
                    buffer.position(buffer.position() + rowStride - length);
                }
            }
        }
    
        return output;
    }
    
        2
  •  2
  •   Yessy    4 年前

    RenderScript支持YUV_420_888作为ScriptIntrinsicYuvToRGB的源代码

    1. 创建分配(&A);ScriptIntrinsicYuvToRGB

      RenderScript renderScript = RenderScript.create(this);
      ScriptIntrinsicYuvToRGB mScriptIntrinsicYuvToRGB = ScriptIntrinsicYuvToRGB.create(renderScript, Element.YUV(renderScript));
      Allocation mAllocationInYUV = Allocation.createTyped(renderScript, new Type.Builder(renderScript, Element.YUV(renderScript)).setYuvFormat(ImageFormat.YUV_420_888).setX(480).setY(640).create(), Allocation.USAGE_IO_INPUT | Allocation.USAGE_SCRIPT);
      Allocation mAllocationOutRGB = Allocation.createTyped(renderScript, Type.createXY(renderScript, Element.RGBA_8888(renderScript), 480, 640), Allocation.USAGE_SCRIPT | Allocation.USAGE_IO_OUTPUT);
      
    2. 设置分配。getSurface()从摄像机接收图像数据

      final CaptureRequest.Builder captureRequest = session.getDevice().createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
      captureRequest.addTarget(mAllocationInYUV.getSurface());
      
    3. 输出到TextureView、ImageReader或SurfaceView

      mAllocationOutRGB.setSurface(new Surface(mTextureView.getSurfaceTexture()));
      mAllocationInYUV.setOnBufferAvailableListener(new Allocation.OnBufferAvailableListener() {
          @Override
          public void onBufferAvailable(Allocation a) {
              a.ioReceive();
              mScriptIntrinsicYuvToRGB.setInput(a);
              mScriptIntrinsicYuvToRGB.forEach(mAllocationOutRGB);
              mAllocationOutRGB.ioSend();
          }
      });
      
        3
  •  1
  •   Alex Cohn    7 年前

    没有直接的方法将YUV_420_888摄像机帧复制到RS分配中。实际上,截至今天,Renderscript does not support this format .

    如果您知道,在引擎盖下,您的帧是NV21或YV12-您可以将整个ByteBuffer复制到一个数组,并将其传递给RS分配。