掌握Android图像显示原理中(二)

5,158

接着上一篇《掌握Android图像显示原理中(一)》

Skia

Skia是谷歌开源的一款跨平台的2D图形引擎,目前谷歌的Chrome浏览器、Android、Flutter、以及火狐浏览器、火狐操作系统和其它许多产品都使用它作为图形引擎,它作为Android系统第三方软件,放在external/skia/ 目录下。虽然Android从4.0开始默认开启了硬件加速,但不代表Skia的作用就不大了,其实Skia在Android中的地位是越来越重要了,从Android 8开始,我们可以选择使用Skia进行硬件加速,Android 9开始就默认使用Skia来进行硬件加速。Skia的硬件加速主要是通过 copybit 模块调用OpenGL或者SKia来实现。

在这里插入图片描述

由于Skia的硬件加速也是通过Copybit模块调用的OpenGL或者Vulkan接口,所以我们这儿只说说Skia通过cpu绘制的,也就是软绘的方式。还是老规则,先看看Skia要如何使用

如何使用Skia?

OpenGL ES的使用要配合EGL,需要初始化Display,surface,context等,用法还是比较繁琐的,Skia在使用上就方便很多了。掌握Skia绘制三要素:画板SKCanvas 、画纸SiBitmap、画笔Skpaint,我们就能很轻松的用Skia来绘制图形。

下面详细的解释Skia的绘图三要素

  1. SKBitmap用来存储图形数据,它封装了与位图相关的一系列操作
SkBitmap bitmap = new SkBitmap();
//设置位图格式及宽高
bitmap->setConfig(SkBitmap::kRGB_565_Config,800,480);
//分配位图所占空间
bitmap->allocPixels();
  1. SKCanvas 封装了所有画图操作的函数,通过调用这些函数,我们就能实现绘制操作。
//使用前传入bitmap
SkCanvas canvas(bitmap);
//移位,缩放,旋转,变形操作
translate(SkiaScalar dx, SkiaScalar dy);
scale(SkScalar sx, SkScalar sy);
rotate(SkScalar degrees);
skew(SkScalar sx, SkScalar sy);
//绘制操作
drawARGB(u8 a, u8 r, u8 g, u8 b....)  //给定透明度以及红,绿,兰3色,填充整个可绘制区域。
drawColor(SkColor color...) //给定颜色color, 填充整个绘制区域。
drawPaint(SkPaint& paint) //用指定的画笔填充整个区域。
drawPoint(...)//根据各种不同参数绘制不同的点。
drawLine(x0, y0, x1, y1, paint) //画线,起点(x0, y0), 终点(x1, y1), 使用paint作为画笔。
drawRect(rect, paint) //画矩形,矩形大小由rect指定,画笔由paint指定。
drawRectCoords(left, top, right, bottom, paint),//给定4个边界画矩阵。
drawOval(SkRect& oval, SkPaint& paint) //画椭圆,椭圆大小由oval矩形指定。
//……其他操作
  1. Skpaint用来设置绘制内容的风格,样式,颜色等信息
setAntiAlias: 设置画笔的锯齿效果。 
setColor: 设置画笔颜色 
setARGB:  设置画笔的a,r,p,g值。 
setAlpha:  设置Alpha值 
setTextSize: 设置字体尺寸。 
setStyle:  设置画笔风格,空心或者实心。 
setStrokeWidth: 设置空心的边框宽度。 
getColor:  得到画笔的颜色 
getAlpha:  得到画笔的Alpha值。 

我们看一个完整的使用Demo

void draw() {
    SkBitmap bitmap = new SkBitmap();
	//设置位图格式及宽高
	bitmap->setConfig(SkBitmap::kRGB_565_Config,800,480);
	//分配位图所占空间
	bitmap->allocPixels();
    //使用前传入bitmap
	SkCanvas canvas(bitmap);
    //定义画笔
    SkPaint paint1, paint2, paint3;

    paint1.setAntiAlias(true);
    paint1.setColor(SkColorSetRGB(255, 0, 0));
    paint1.setStyle(SkPaint::kFill_Style);

    paint2.setAntiAlias(true);
    paint2.setColor(SkColorSetRGB(0, 136, 0));
    paint2.setStyle(SkPaint::kStroke_Style);
    paint2.setStrokeWidth(SkIntToScalar(3));

    paint3.setAntiAlias(true);
    paint3.setColor(SkColorSetRGB(136, 136, 136));

    sk_sp<SkTextBlob> blob1 =
            SkTextBlob::MakeFromString("Skia!", SkFont(nullptr, 64.0f, 1.0f, 0.0f));
    sk_sp<SkTextBlob> blob2 =
            SkTextBlob::MakeFromString("Skia!", SkFont(nullptr, 64.0f, 1.5f, 0.0f));

    canvas->clear(SK_ColorWHITE);
    canvas->drawTextBlob(blob1.get(), 20.0f, 64.0f, paint1);
    canvas->drawTextBlob(blob1.get(), 20.0f, 144.0f, paint2);
    canvas->drawTextBlob(blob2.get(), 20.0f, 224.0f, paint3);
}

这个Demo的效果如下

在这里插入图片描述

了解了Skia如何使用,我们接着看两个场景:Skia进行软件绘制,Flutter界面绘制

Skia进行软件绘制

在上面我讲了通过使用OpenGL渲染的硬件绘制方式,这里会接着讲使用Skia渲染的软件绘制方式,虽然Android默认开启了硬件加速,但是由于硬件加速会有耗电和内存的问题,一些系统应用和常驻应用依然是使用的软件绘制的方式,软绘入口还是在draw方法中。

//文件-->/frameworks/base/core/java/android/view/ViewRootImpl.java
private void performDraw() {
    //……
    draw(fullRedrawNeeded);
    //……
}

private void draw(boolean fullRedrawNeeded) {
    Surface surface = mSurface;
    if (!surface.isValid()) {
        return;
    }

    //……

    if (!dirty.isEmpty() || mIsAnimating || accessibilityFocusDirty) {
        if (!dirty.isEmpty() || mIsAnimating || accessibilityFocusDirty) {
            if (mAttachInfo.mThreadedRenderer != null && mAttachInfo.mThreadedRenderer.isEnabled()) {
                
                //……

                //硬件渲染
                mAttachInfo.mThreadedRenderer.draw(mView, mAttachInfo, this);

            } else {
                
                //……

                //软件渲染
                if (!drawSoftware(surface, mAttachInfo, xOffset, yOffset, scalingRequired, dirty)) {
                    return;
                }
            }
        }

        //……
    }

    //……
}

我们来看看drawSoftware函数的实现

private boolean drawSoftware(Surface surface, AttachInfo attachInfo, int xoff, int yoff,
        boolean scalingRequired, Rect dirty) {

    // Draw with software renderer.
    final Canvas canvas;

    //……

    canvas = mSurface.lockCanvas(dirty);
    
    //……
        
    mView.draw(canvas);
    
    //……
    
    surface.unlockCanvasAndPost(canvas);
    
    //……
    
    return true;
}

drawSoftware函数的流程主要为三步

  1. 通过mSurface.lockCanvas获取Canvas
  2. 通过draw方法,将根View及其子View遍历绘制到Canvas上
  3. 通过surface.unlockCanvasAndPost将绘制内容提交给surfaceFlinger进行合成

Lock Surface

我们先来看第一步,这个Canvas对应着Native层的SKCanvas。

//文件-->/frameworks/base/core/java/android/view/Surface.java
public Canvas lockCanvas(Rect inOutDirty)
    throws Surface.OutOfResourcesException, IllegalArgumentException {
    synchronized (mLock) {
        checkNotReleasedLocked();
        if (mLockedObject != 0) {           
            throw new IllegalArgumentException("Surface was already locked");
        }
        mLockedObject = nativeLockCanvas(mNativeObject, mCanvas, inOutDirty);
        return mCanvas;
    }
}

lockCanvas函数中通过JNI函数nativeLockCanvas,创建Nativce层的Canvas,nativeLockCanvas的入参mNativeObject对应着Native层的Surface,关于Surface和Buffer的知识,在下一篇图形缓冲区中会详细简介,这里不做太多介绍。我们直接着看nativeLockCanvas的实现。

static jlong nativeLockCanvas(JNIEnv* env, jclass clazz,
        jlong nativeObject, jobject canvasObj, jobject dirtyRectObj) {
    sp<Surface> surface(reinterpret_cast<Surface *>(nativeObject));

    if (!isSurfaceValid(surface)) {
        doThrowIAE(env);
        return 0;
    }

    Rect dirtyRect(Rect::EMPTY_RECT);
    Rect* dirtyRectPtr = NULL;

    if (dirtyRectObj) {
        dirtyRect.left   = env->GetIntField(dirtyRectObj, gRectClassInfo.left);
        dirtyRect.top    = env->GetIntField(dirtyRectObj, gRectClassInfo.top);
        dirtyRect.right  = env->GetIntField(dirtyRectObj, gRectClassInfo.right);
        dirtyRect.bottom = env->GetIntField(dirtyRectObj, gRectClassInfo.bottom);
        dirtyRectPtr = &dirtyRect;
    }

    ANativeWindow_Buffer outBuffer;
    //1,获取用来存储图形绘制的buffer
    status_t err = surface->lock(&outBuffer, dirtyRectPtr);
    if (err < 0) {
        const char* const exception = (err == NO_MEMORY) ?
                OutOfResourcesException :
                "java/lang/IllegalArgumentException";
        jniThrowException(env, exception, NULL);
        return 0;
    }

    SkImageInfo info = SkImageInfo::Make(outBuffer.width, outBuffer.height,
                                         convertPixelFormat(outBuffer.format),
                                         outBuffer.format == PIXEL_FORMAT_RGBX_8888
                                                 ? kOpaque_SkAlphaType : kPremul_SkAlphaType,
                                         GraphicsJNI::defaultColorSpace());

    SkBitmap bitmap;
    ssize_t bpr = outBuffer.stride * bytesPerPixel(outBuffer.format);
    bitmap.setInfo(info, bpr);
   
    if (outBuffer.width > 0 && outBuffer.height > 0) {
         //将上一个buffer里的图形数据复制到当前bitmap中
        bitmap.setPixels(outBuffer.bits);
    } else {
        // be safe with an empty bitmap.
        bitmap.setPixels(NULL);
    }

    //2,创建一个SKCanvas
    Canvas* nativeCanvas = GraphicsJNI::getNativeCanvas(env, canvasObj);
    //3,给SKCanvas设置Bitmap
    nativeCanvas->setBitmap(bitmap);
    //如果指定了脏区,则设定脏区的区域
    if (dirtyRectPtr) {
        nativeCanvas->clipRect(dirtyRect.left, dirtyRect.top,
                dirtyRect.right, dirtyRect.bottom, SkClipOp::kIntersect);
    }

    if (dirtyRectObj) {
        env->SetIntField(dirtyRectObj, gRectClassInfo.left,   dirtyRect.left);
        env->SetIntField(dirtyRectObj, gRectClassInfo.top,    dirtyRect.top);
        env->SetIntField(dirtyRectObj, gRectClassInfo.right,  dirtyRect.right);
        env->SetIntField(dirtyRectObj, gRectClassInfo.bottom, dirtyRect.bottom);
    }

    sp<Surface> lockedSurface(surface);
    lockedSurface->incStrong(&sRefBaseOwner);
    return (jlong) lockedSurface.get();
}

nativeLockCanvas主要做了这几件事情

  1. 通过surface->lock函数获取绘制用的Buffer
  2. 根据Buffer信息创建SKBitmap
  3. 根据SKBitmap,创建并初始化SKCanvas

通过nativeLockCanvas,我们就创建好了SKCanvas了,并且设置了可以绘制图形的bitmap,此时我们就可以通过SKCanvas往bitmap里面绘制图形,mView.draw()函数,就做了这件事情。

绘制

我们接着看看View中的draw()函数

//文件-->/frameworks/base/core/java/android/view/View.java
public void draw(Canvas canvas) {
    final int privateFlags = mPrivateFlags;
    final boolean dirtyOpaque = (privateFlags & PFLAG_DIRTY_MASK) == PFLAG_DIRTY_OPAQUE &&
            (mAttachInfo == null || !mAttachInfo.mIgnoreDirtyState);
    mPrivateFlags = (privateFlags & ~PFLAG_DIRTY_MASK) | PFLAG_DRAWN;

    int saveCount;
    //1,绘制背景
    if (!dirtyOpaque) {
        drawBackground(canvas);
    }

    final int viewFlags = mViewFlags;
    boolean horizontalEdges = (viewFlags & FADING_EDGE_HORIZONTAL) != 0;
    boolean verticalEdges = (viewFlags & FADING_EDGE_VERTICAL) != 0;
    if (!verticalEdges && !horizontalEdges) {
        // 2,绘制当前view的图形
        if (!dirtyOpaque) onDraw(canvas);

        // 3,绘制子view的图形
        dispatchDraw(canvas);

        drawAutofilledHighlight(canvas);

        // Overlay is part of the content and draws beneath Foreground
        if (mOverlay != null && !mOverlay.isEmpty()) {
            mOverlay.getOverlayView().dispatchDraw(canvas);
        }

        //4,绘制decorations,如滚动条,前景等 Step 6, draw decorations (foreground, scrollbars)
        onDrawForeground(canvas);

        // 5,绘制焦点的高亮
        drawDefaultFocusHighlight(canvas);

        if (debugDraw()) {
            debugDrawFocus(canvas);
        }

        // we're done...
        return;
    }

    //……
}

draw函数中做了这几件事情

  1. 绘制背景
  2. 绘制当前view
  3. 遍历绘制子view
  4. 绘制前景

我们可以看看Canvas里的绘制方法,这些绘制方法都是JNI方法,并且一一对应着SKCanvas中的绘制方法

//文件-->/frameworks/base/graphics/java/android/graphics/Canvas.java

//……
private static native void nDrawBitmap(long nativeCanvas, int[] colors, int offset, int stride,
            float x, float y, int width, int height, boolean hasAlpha, long nativePaintOrZero);

private static native void nDrawColor(long nativeCanvas, int color, int mode);

private static native void nDrawPaint(long nativeCanvas, long nativePaint);

private static native void nDrawPoint(long canvasHandle, float x, float y, long paintHandle);

private static native void nDrawPoints(long canvasHandle, float[] pts, int offset, int count,
                                       long paintHandle);

private static native void nDrawLine(long nativeCanvas, float startX, float startY, float stopX,
                                     float stopY, long nativePaint);

private static native void nDrawLines(long canvasHandle, float[] pts, int offset, int count,
                                      long paintHandle);

private static native void nDrawRect(long nativeCanvas, float left, float top, float right,
                                     float bottom, long nativePaint);

private static native void nDrawOval(long nativeCanvas, float left, float top, float right,
                                     float bottom, long nativePaint);

private static native void nDrawCircle(long nativeCanvas, float cx, float cy, float radius,
                                       long nativePaint);

private static native void nDrawArc(long nativeCanvas, float left, float top, float right,
                                    float bottom, float startAngle, float sweep, boolean useCenter, long nativePaint);

private static native void nDrawRoundRect(long nativeCanvas, float left, float top, float right,
                                          float bottom, float rx, float ry, long nativePaint);
//……

Post Surface

软件绘制的最后一步,通过surface.unlockCanvasAndPost将绘制内容提交给surfaceFlinger绘制,将绘制出来的图形提交给SurfaceFlinger,然后SurfaceFlinger作为消费者处理图形后,我们的界面就显示出来了。

public void unlockCanvasAndPost(Canvas canvas) {
    synchronized (mLock) {
        checkNotReleasedLocked();

        if (mHwuiContext != null) {
            mHwuiContext.unlockAndPost(canvas);
        } else {
            unlockSwCanvasAndPost(canvas);
        }
    }
}

private void unlockSwCanvasAndPost(Canvas canvas) {
    if (canvas != mCanvas) {
        throw new IllegalArgumentException("canvas object must be the same instance that "
                                           + "was previously returned by lockCanvas");
    }
    if (mNativeObject != mLockedObject) {
        Log.w(TAG, "WARNING: Surface's mNativeObject (0x" +
              Long.toHexString(mNativeObject) + ") != mLockedObject (0x" +
              Long.toHexString(mLockedObject) +")");
    }
    if (mLockedObject == 0) {
        throw new IllegalStateException("Surface was not locked");
    }
    try {
        nativeUnlockCanvasAndPost(mLockedObject, canvas);
    } finally {
        nativeRelease(mLockedObject);
        mLockedObject = 0;
    }
}

这里调用了Native函数nativeUnlockCanvasAndPost,我们接着往下看。

static void nativeUnlockCanvasAndPost(JNIEnv* env, jclass clazz,
        jlong nativeObject, jobject canvasObj) {
    sp<Surface> surface(reinterpret_cast<Surface *>(nativeObject));
    if (!isSurfaceValid(surface)) {
        return;
    }

    // detach the canvas from the surface
    Canvas* nativeCanvas = GraphicsJNI::getNativeCanvas(env, canvasObj);
    nativeCanvas->setBitmap(SkBitmap());

    // unlock surface
    status_t err = surface->unlockAndPost();
    if (err < 0) {
        doThrowIAE(env);
    }
}

在这里,**surface->unlockAndPost()**函数就会将Skia绘制出来的图像传递给SurfaceFlinger进行合成。通过skia进行软件绘制的流程已经讲完了,至于如何通过Surface获取缓冲区,在缓冲区绘制完数据后,**surface->unlockAndPost()**又如何通知SurfaceFlinger,这一点在下一篇文章的图形缓冲区中会详细的讲解。

可以看到,Skia软件绘制的流程比硬件绘制要简单很多,我们接着看看Skia进行Flutter绘制的案例。

Skia进行Flutter的界面绘制

在讲解Flutter如何通过Skia生产图像之前,先简单介绍一下Flutter,Flutter的架构分为Framework层,Engine层和Embedder三层。

  • Framework层使用dart语言实现,包括UI,文本,图片,按钮等Widgets,渲染,动画,手势等。

  • Engine使用C++实现,主要包括渲染引擎Skia, Dart虚拟机和文字排版Tex等模块。

  • Embedder是一个嵌入层,通过该层把Flutter嵌入到各个平台上去,Embedder的主要工作包括渲染Surface设置, 线程设置,以及插件等

C:\Users\Administrator\OneDrive\技术笔记\素材\cbef8449301eaff5864272c3c609bee2.png

了解了Flutter的架构,我们在接着了解Flutter显示一个界面的流程。我们知道在Android中,显示一个界面需要将XML界面布局解析成ViewGroup,然后再经过测量Measure,布局Layout和绘制Draw的流程。Flutter和Android的显示不太一样,它会将通过Dart语言编写的Widget界面布局转换成ElementTree和Render ObjectTree。ElementTree相当于是ViewGroup,Render ObjectTree相当于是经过Measure和Layout流程之后的ViewGroup。这种模式在很多场景上都有使用,比如Webview,在渲染界面时,也会创建一颗Dom树,render树和RenderObject,这样的好处是可以通过Diff比较改变过的组件,然后渲染时,只对改变过的组件做渲染,同时对跨平台友好,可以通过这种树的形式来抽象出不同平台的公共部分。

在这里插入图片描述

讲完了上面两个背景,我们直接来看Flutter是如何使用Skia来绘制界面的。

下面是一个Flutter页面的Demo

import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

这个页面是一个WidgetTree,相当于我们Activity的xml,widget树会转换成ElementTree和RenderObjectTree,我们看看入口函数runApp时如何进行树的转换的。

//文件-->/packages/flutter/lib/src/widgets
void runApp(Widget app) {
  WidgetsFlutterBinding.ensureInitialized()
    ..scheduleAttachRootWidget(app)
    ..scheduleWarmUpFrame();
}

void scheduleAttachRootWidget(Widget rootWidget) {
    Timer.run(() {
        attachRootWidget(rootWidget);
    });
}

void attachRootWidget(Widget rootWidget) {
    _readyToProduceFrames = true;
    _renderViewElement = RenderObjectToWidgetAdapter<RenderBox>(
        container: renderView,
        debugShortDescription: '[root]',
        child: rootWidget,
    ).attachToRenderTree(buildOwner, renderViewElement as RenderObjectToWidgetElement<RenderBox>);
}

接着看attachToRenderTree函数

RenderObjectToWidgetElement<T> attachToRenderTree(BuildOwner owner, [RenderObjectToWidgetElement<T> element]) {
    if (element == null) {
      owner.lockState(() {
        element = createElement();  //创建rootElement
        element.assignOwner(owner); //绑定BuildOwner
      });
      owner.buildScope(element, () { //子widget的初始化从这里开始
        element.mount(null, null);  // 初始化子Widget前,先执行rootElement的mount方法
      });
    } else {
      ...
    }
    return element;
  }

void mount(Element parent, dynamic newSlot) {
  super.mount(parent, newSlot);
  _renderObject = widget.createRenderObject(this);
  attachRenderObject(newSlot);
  _dirty = false;
}

从代码中可以看到,Widget都被转换成了Element,Element接着调用了mount方法,在mount方法中,可以看到Widget又被转换成了RenderObject,此时Widget Tree的ElementTree和RenderObject便都生成完了。

前面提到了RenderObject类似于经过了Measure和Layout流程的ViewGroup,RenderObject的Measure和Layout就不在这儿说了,那么还剩一个流程Draw流程,同样是在RenderObject中进行的,它的入口在RenderObject的paint函数中。

// 绘制入口,从 view 根节点开始,逐个绘制所有子节点
@override
  void paint(PaintingContext context, Offset offset) {
    if (child != null)
      context.paintChild(child, offset);
  }

可以看到,RenderObject通过PaintingContext来进行了图形的绘制,我们接着来了解一下PaintingContext是什么。

//文件-->/packages/flutter/lib/src/rendering/object.dart

import 'dart:ui' as ui show PictureRecorder;

class PaintingContext extends ClipContext {
  @protected
  PaintingContext(this._containerLayer, this.estimatedBounds)
 
  final ContainerLayer _containerLayer;
  final Rect estimatedBounds;
  
  PictureLayer _currentLayer;
  ui.PictureRecorder _recorder;
  Canvas _canvas;
 
  @override
  Canvas get canvas {
    if (_canvas == null)
      _startRecording();
    return _canvas;
  }
 
  void _startRecording() {
    _currentLayer = PictureLayer(estimatedBounds);
    _recorder = ui.PictureRecorder();
    _canvas = Canvas(_recorder);
    _containerLayer.append(_currentLayer);
  }
  
   void stopRecordingIfNeeded() {
    if (!_isRecording)
      return;
    _currentLayer.picture = _recorder.endRecording();
    _currentLayer = null;
    _recorder = null;
    _canvas = null;
}

可以看到,PaintingContext是绘制的上下文,前面讲OpenGL进行硬件加速时提到的CanvasContext,它也是绘制的上下文,里面封装了Skia,Opengl或者Vulkan的渲染管线。这里的PaintingContext则封装了Skia。

我们可以通过CanvasContext的get canvas函数获取Canvas,它调用了_startRecording函数中,函数中创建了PictureRecorder和Canvas,这两个类都是位于dart:ui库中,dart:ui位于engine层,在前面架构中提到,Flutter分为Framewrok,Engine和embened三层,Engine中包含了Skia,dart虚拟机和Text。dart:ui就是位于Engine层的。

我们接着去Engine层的代码看看Canvas的实现。

//文件-->engine-master\lib\ui\canvas.dart  
Canvas(PictureRecorder recorder, [ Rect? cullRect ]) : assert(recorder != null) { // ignore: unnecessary_null_comparison
    if (recorder.isRecording)
      throw ArgumentError('"recorder" must not already be associated with another Canvas.');
    _recorder = recorder;
    _recorder!._canvas = this;
    cullRect ??= Rect.largest;
    _constructor(recorder, cullRect.left, cullRect.top, cullRect.right, cullRect.bottom);
  }
void _constructor(PictureRecorder recorder,
                  double left,
                  double top,
                  double right,
                  double bottom) native 'Canvas_constructor';

这里Canvas调用了Canvas_constructor这一个native方法,我们接着看这个native方法的实现。

//文件-->engine-master\lib\ui\painting\engine.cc
static void Canvas_constructor(Dart_NativeArguments args) {
  UIDartState::ThrowIfUIOperationsProhibited();
  DartCallConstructor(&Canvas::Create, args);
}
fml::RefPtr<Canvas> Canvas::Create(PictureRecorder* recorder,
                                   double left,
                                   double top,
                                   double right,
                                   double bottom) {
  if (!recorder) {
    Dart_ThrowException(
        ToDart("Canvas constructor called with non-genuine PictureRecorder."));
    return nullptr;
  }
  fml::RefPtr<Canvas> canvas = fml::MakeRefCounted<Canvas>(
      recorder->BeginRecording(SkRect::MakeLTRB(left, top, right, bottom)));
  recorder->set_canvas(canvas);
  return canvas;
}

Canvas::Canvas(SkCanvas* canvas) : canvas_(canvas) {}

可以看到,这里通过PictureRecorder->BeginRecording创建了SKCanvas,这其实是SKCanvas的另外一种使用方式,这里我简单的介绍一个使用demo。

Picture createSolidRectanglePicture(
  Color color, double width, double height)
{

  PictureRecorder recorder = PictureRecorder();
  Canvas canvas = Canvas(recorder);

  Paint paint = Paint();
  paint.color = color;

  canvas.drawRect(Rect.fromLTWH(0, 0, width, height), paint);
  return recorder.endRecording();
}

这个demo的效果如下图,它创建Skia的方式就和Flutter创建Skia的方式是一样的。 在这里插入图片描述

此时,我们的SKCanvas创建好了,并且直接通过PaintingContext的get canvas函数就能获取到,那么获取到SKCanvas后直接调用Canvas的绘制api,就可以将图像绘制出来了。

Flutter界面显示的全流程是比较复杂的,Flutter是完全是自建的一套图像显示流程,无法通过Android的SurfaceFlinger进行图像合成,也无法使用Android的Gralloc模块分配图像缓冲区,所以它需要有自己的图像生产者,有自己的图形消费者,也有自己的图形缓冲区,这里面就有非常多的流程,比如如何接收VSync,如何处理及合成Layer,如何创建图像缓冲区,这里只是对Flutter的图像生产者的部分做了一个初步的介绍,关于Flutter更深入一步的细节,就不在这里继续讲解了。后面我会专门写一系列文章来详细讲解Flutter。

Vulkan

与OpenGL相比,Vulkan可以更详细的向显卡描述你的应用程序打算做什么,从而可以获得更好的性能和更小的驱动开销,作为OpenGL的替代者,它设计之初就是为了跨平台实现的,可以同时在Windows、Linux和Android开发。甚至在Mac OS系统上运行。Android在7.0开始,便增加了对Vulkan的支持,Vulkan一定是未来的趋势,因为它比OpenGL的性能更好更强大。下面我们就了解一下,如何使用Vulkan来生产图像。

如何使用Vulkan?

Vulkan的使用和OpenGL类似,同样是三步:初始化,绘制,提交buffer下面来看一下具体的流程

1,初始化Vulkan实例,物理设备和任务队列以及Surface

  • 创建Instances实例
VkInstanceCreateInfo instance_create_info = { 
  VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO, 
  nullptr, 
  0, 
  &application_info, 
  0, 
  nullptr, 
  static_cast<uint32_t>(desired_extensions.size()), 
  desired_extensions.size() > 0 ? &desired_extensions[0] : nullptr 
};

VkInstance inst;
VkResult result = vkCreateInstance( &instance_create_info, nullptr, &inst ); 
  • 初始化物理设备,也就是我们的显卡设备,Vulkna的设计是支持多GPU的,这里选择第一个设备就行了。
uint32_t extensions_count = 0; 
VkResult result = VK_SUCCESS; 

//获取所有可用物理设备,并选择第一个
result = vkEnumerateDeviceExtensionProperties( physical_device, nullptr, &extensions_count, &available_extensions[0]); 
if( (result != VK_SUCCESS) || 
    (extensions_count == 0) ) { 
  std::cout << "Could not get the number of device extensions." << std::endl; 
  return false; 
}
  • 获取queue,Vulkan的所有操作,从绘图到上传纹理,都需要将命令提交到队列中
uint32_t queue_families_count = 0; 

//获取队列簇,并选择第一个
queue_families.resize( queue_families_count ); 
vkGetPhysicalDeviceQueueFamilyProperties( physical_device, &queue_families_count, &queue_families[0] ); 
if( queue_families_count == 0 ) { 
  std::cout << "Could not acquire properties of queue families." << std::endl; 
  return false; 
} 
  • 初始化逻辑设备,在选择要使用的物理设备之后,我们需要设置一个逻辑设备用于交互。
VkResult result = vkCreateDevice( physical_device, &device_create_info, nullptr, &logical_device ); 
if( (result != VK_SUCCESS) || 
    (logical_device == VK_NULL_HANDLE) ) { 
  std::cout << "Could not create logical device." << std::endl; 
  return false; 
} 

return true;
  • 上述初始完毕后,接着初始化Surface,然后我们就可以使用Vulkan进行绘制了
#ifdef VK_USE_PLATFORM_WIN32_KHR 

//创建WIN32的surface,如果是Android,需要使用VkAndroidSurfaceCreateInfoKHR
VkWin32SurfaceCreateInfoKHR surface_create_info = { 
  VK_STRUCTURE_TYPE_WIN32_SURFACE_CREATE_INFO_KHR, 
  nullptr, 
  0, 
  window_parameters.HInstance, 
  window_parameters.HWnd 
}; 

VkResult result = vkCreateWin32SurfaceKHR( instance, &surface_create_info, nullptr, &presentation_surface );

2,通过vkCmdDraw函数进行图像绘制

void vkCmdDraw(
    //在Vulkan中,像绘画命令、内存转换等操作并不是直接通过方法调用去完成的,而是需要把所有的操作放在Command Buffer里
    VkCommandBuffer commandBuffer,
    uint32_t vertexCount, //顶点数量
    uint32_t instanceCount, // 要画的instance数量,没有:置1
    uint32_t firstVertex,// vertex buffer中第一个位置 和 vertex Shader 里gl_vertexIndex 相关。
    uint32_t firstInstance);// 同firstVertex 类似。

3,提交buffer

if (vkQueueSubmit(graphicsQueue, 1, &submitInfo, VK_NULL_HANDLE) != VK_SUCCESS) {
    throw std::runtime_error("failed to submit draw command buffer!");
}

我在这里比较浅显的介绍了Vulkan的用法,但上面介绍的只是Vulkan的一点皮毛,Vulkan的使用比OpenGL要复杂的很多,机制也复杂很多,如果想进一步了解Vulkan还是得专门去深入研究。虽然只介绍了一点皮毛,但已经可以让我们去了解Vulkan这一图像生产者,是如何在Android系统中生产图像的,下面就来看看吧。

Vulkan进行硬件加速

在前面讲OpenGL 进行硬件加速时,提到了CanvasContext,它会根据渲染的类型选择不同的渲染管线,Android是通过Vulkan或者还是通过OpenGL渲染,主要是CanvasContext里选择的渲染管线的不同。

CanvasContext* CanvasContext::create(RenderThread& thread,
        bool translucent, RenderNode* rootRenderNode, IContextFactory* contextFactory) {

    auto renderType = Properties::getRenderPipelineType();

    switch (renderType) {
        case RenderPipelineType::OpenGL:
            return new CanvasContext(thread, translucent, rootRenderNode, contextFactory,
                    std::make_unique<OpenGLPipeline>(thread));
        case RenderPipelineType::SkiaGL:
            return new CanvasContext(thread, translucent, rootRenderNode, contextFactory,
                    std::make_unique<skiapipeline::SkiaOpenGLPipeline>(thread));
        case RenderPipelineType::SkiaVulkan:
            return new CanvasContext(thread, translucent, rootRenderNode, contextFactory,
                                std::make_unique<skiapipeline::SkiaVulkanPipeline>(thread));
        default:
            LOG_ALWAYS_FATAL("canvas context type %d not supported", (int32_t) renderType);
            break;
    }
    return nullptr;
}

我们这里直接看SkiaVulkanPipeline。

//文件->/frameworks/base/libs/hwui/pipeline/skia/SkiaVulkanPipeline.cpp
SkiaVulkanPipeline::SkiaVulkanPipeline(renderthread::RenderThread& thread)
        : SkiaPipeline(thread), mVkManager(thread.vulkanManager()) {}

SkiaVulkanPipeline的构造函数中初始化了VulkanManager,VulkanManager是对Vulkan使用的封装,和前面讲到的OpenGLPipeline中的EglManager类似。我们看一下VulkanManager的初始化函数。

//文件-->/frameworks/base/libs/hwui/renderthread/VulkanManager.cpp
void VulkanManager::initialize() {
    if (hasVkContext()) {
        return;
    }

    auto canPresent = [](VkInstance, VkPhysicalDevice, uint32_t) { return true; };
    mBackendContext.reset(GrVkBackendContext::Create(vkGetInstanceProcAddr, vkGetDeviceProcAddr,
                                                     &mPresentQueueIndex, canPresent));
	//……
}

初始化函数中我们主要关注GrVkBackendContext::Create方法。

// Create the base Vulkan objects needed by the GrVkGpu object
const GrVkBackendContext* GrVkBackendContext::Create(uint32_t* presentQueueIndexPtr,
                                                     CanPresentFn canPresent,
                                                     GrVkInterface::GetProc getProc) {
    //……

    const VkInstanceCreateInfo instance_create = {
        VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO,    // sType
        nullptr,                                   // pNext
        0,                                         // flags
        &app_info,                                 // pApplicationInfo
        (uint32_t) instanceLayerNames.count(),     // enabledLayerNameCount
        instanceLayerNames.begin(),                // ppEnabledLayerNames
        (uint32_t) instanceExtensionNames.count(), // enabledExtensionNameCount
        instanceExtensionNames.begin(),            // ppEnabledExtensionNames
    };

    ACQUIRE_VK_PROC(CreateInstance, VK_NULL_HANDLE, VK_NULL_HANDLE);
    //1,创建Vulkan实例
    err = grVkCreateInstance(&instance_create, nullptr, &inst);
    if (err < 0) {
        SkDebugf("vkCreateInstance failed: %d\n", err);
        return nullptr;
    }

    

    uint32_t gpuCount;
    //2,查询可用物理设备
    err = grVkEnumeratePhysicalDevices(inst, &gpuCount, nullptr);
    if (err) {
        //……
    }
    //……
    gpuCount = 1;
    //3,选择物理设备
    
    err = grVkEnumeratePhysicalDevices(inst, &gpuCount, &physDev);
    if (err) {
        //……
    }

    //4,查询队列簇
    uint32_t queueCount;
    grVkGetPhysicalDeviceQueueFamilyProperties(physDev, &queueCount, nullptr);
    if (!queueCount) {
        //……
        return nullptr;
    }

    SkAutoMalloc queuePropsAlloc(queueCount * sizeof(VkQueueFamilyProperties));
    // now get the actual queue props
    VkQueueFamilyProperties* queueProps = (VkQueueFamilyProperties*)queuePropsAlloc.get();
	//5,选择队列簇
    grVkGetPhysicalDeviceQueueFamilyProperties(physDev, &queueCount, queueProps);
    
    //……

    // iterate to find the graphics queue
    const VkDeviceCreateInfo deviceInfo = {
        VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO,    // sType
        nullptr,                                 // pNext
        0,                                       // VkDeviceCreateFlags
        queueInfoCount,                          // queueCreateInfoCount
        queueInfo,                               // pQueueCreateInfos
        (uint32_t) deviceLayerNames.count(),     // layerCount
        deviceLayerNames.begin(),                // ppEnabledLayerNames
        (uint32_t) deviceExtensionNames.count(), // extensionCount
        deviceExtensionNames.begin(),            // ppEnabledExtensionNames
        &deviceFeatures                          // ppEnabledFeatures
    };
	//6,创建逻辑设备
    err = grVkCreateDevice(physDev, &deviceInfo, nullptr, &device);
    if (err) {
        SkDebugf("CreateDevice failed: %d\n", err);
        grVkDestroyInstance(inst, nullptr);
        return nullptr;
    }

    auto interface =
        sk_make_sp<GrVkInterface>(getProc, inst, device, extensionFlags);
    if (!interface->validate(extensionFlags)) {
        SkDebugf("Vulkan interface validation failed\n");
        grVkDeviceWaitIdle(device);
        grVkDestroyDevice(device, nullptr);
        grVkDestroyInstance(inst, nullptr);
        return nullptr;
    }

    VkQueue queue;
    grVkGetDeviceQueue(device, graphicsQueueIndex, 0, &queue);

    GrVkBackendContext* ctx = new GrVkBackendContext();
    ctx->fInstance = inst;
    ctx->fPhysicalDevice = physDev;
    ctx->fDevice = device;
    ctx->fQueue = queue;
    ctx->fGraphicsQueueIndex = graphicsQueueIndex;
    ctx->fMinAPIVersion = kGrVkMinimumVersion;
    ctx->fExtensions = extensionFlags;
    ctx->fFeatures = featureFlags;
    ctx->fInterface.reset(interface.release());
    ctx->fOwnsInstanceAndDevice = true;

    return ctx;
}

可以看到,GrVkBackendContext::Create中所作的事情就是初始化Vulkan,初始化的流程和前面介绍如何使用Vulkan中初始化流程都是一样的,这些都是通用的流程。

初始化完成,我们接着看看Vulkan如何绑定Surface,只有绑定了Surface,我们才能使用Vulkan进行图像绘制。

//文件-->/frameworks/base/libs/hwui/renderthread/VulkanManager.cpp
VulkanSurface* VulkanManager::createSurface(ANativeWindow* window) {
    initialize();

    if (!window) {
        return nullptr;
    }

    VulkanSurface* surface = new VulkanSurface();

    VkAndroidSurfaceCreateInfoKHR surfaceCreateInfo;
    memset(&surfaceCreateInfo, 0, sizeof(VkAndroidSurfaceCreateInfoKHR));
    surfaceCreateInfo.sType = VK_STRUCTURE_TYPE_ANDROID_SURFACE_CREATE_INFO_KHR;
    surfaceCreateInfo.pNext = nullptr;
    surfaceCreateInfo.flags = 0;
    surfaceCreateInfo.window = window;

    VkResult res = mCreateAndroidSurfaceKHR(mBackendContext->fInstance, &surfaceCreateInfo, nullptr,
                                            &surface->mVkSurface);
    if (VK_SUCCESS != res) {
        delete surface;
        return nullptr;
    }

    SkDEBUGCODE(VkBool32 supported; res = mGetPhysicalDeviceSurfaceSupportKHR(
                                            mBackendContext->fPhysicalDevice, mPresentQueueIndex,
                                            surface->mVkSurface, &supported);
                // All physical devices and queue families on Android must be capable of
                // presentation with any
                // native window.
                SkASSERT(VK_SUCCESS == res && supported););

    if (!createSwapchain(surface)) {
        destroySurface(surface);
        return nullptr;
    }

    return surface;
}

可以看到,这个创建了VulkanSurface,并绑定了ANativeWindow,ANativeWindow是Android的原生窗口,在前面介绍OpenGL进行硬件渲染时,也提到过createSurface这个函数,它是在performDraw被执行的,在这里就不重复说了。

接下来就是调用Vulkan的api进行绘制的图像的流程

bool SkiaVulkanPipeline::draw(const Frame& frame, const SkRect& screenDirty, const SkRect& dirty,
                              const FrameBuilder::LightGeometry& lightGeometry,
                              LayerUpdateQueue* layerUpdateQueue, const Rect& contentDrawBounds,
                              bool opaque, bool wideColorGamut,
                              const BakedOpRenderer::LightInfo& lightInfo,
                              const std::vector<sp<RenderNode>>& renderNodes,
                              FrameInfoVisualizer* profiler) {
    sk_sp<SkSurface> backBuffer = mVkSurface->getBackBufferSurface();
    if (backBuffer.get() == nullptr) {
        return false;
    }
    SkiaPipeline::updateLighting(lightGeometry, lightInfo);
    renderFrame(*layerUpdateQueue, dirty, renderNodes, opaque, wideColorGamut, contentDrawBounds,
                backBuffer);
    layerUpdateQueue->clear();

    // Draw visual debugging features
    if (CC_UNLIKELY(Properties::showDirtyRegions ||
                    ProfileType::None != Properties::getProfileType())) {
        SkCanvas* profileCanvas = backBuffer->getCanvas();
        SkiaProfileRenderer profileRenderer(profileCanvas);
        profiler->draw(profileRenderer);
        profileCanvas->flush();
    }

    // Log memory statistics
    if (CC_UNLIKELY(Properties::debugLevel != kDebugDisabled)) {
        dumpResourceCacheUsage();
    }

    return true;
}

最后通过swapBuffers提交绘制内容

void VulkanManager::swapBuffers(VulkanSurface* surface) {
    if (CC_UNLIKELY(Properties::waitForGpuCompletion)) {
        ATRACE_NAME("Finishing GPU work");
        mDeviceWaitIdle(mBackendContext->fDevice);
    }

    SkASSERT(surface->mBackbuffers);
    VulkanSurface::BackbufferInfo* backbuffer =
            surface->mBackbuffers + surface->mCurrentBackbufferIndex;
    GrVkImageInfo* imageInfo;
    SkSurface* skSurface = surface->mImageInfos[backbuffer->mImageIndex].mSurface.get();
    skSurface->getRenderTargetHandle((GrBackendObject*)&imageInfo,
                                     SkSurface::kFlushRead_BackendHandleAccess);
    // Check to make sure we never change the actually wrapped image
    SkASSERT(imageInfo->fImage == surface->mImages[backbuffer->mImageIndex]);

    // We need to transition the image to VK_IMAGE_LAYOUT_PRESENT_SRC_KHR and make sure that all
    // previous work is complete for before presenting. So we first add the necessary barrier here.
    VkImageLayout layout = imageInfo->fImageLayout;
    VkPipelineStageFlags srcStageMask = layoutToPipelineStageFlags(layout);
    VkPipelineStageFlags dstStageMask = VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT;
    VkAccessFlags srcAccessMask = layoutToSrcAccessMask(layout);
    VkAccessFlags dstAccessMask = VK_ACCESS_MEMORY_READ_BIT;

    VkImageMemoryBarrier imageMemoryBarrier = {
            VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,     // sType
            NULL,                                       // pNext
            srcAccessMask,                              // outputMask
            dstAccessMask,                              // inputMask
            layout,                                     // oldLayout
            VK_IMAGE_LAYOUT_PRESENT_SRC_KHR,            // newLayout
            mBackendContext->fGraphicsQueueIndex,       // srcQueueFamilyIndex
            mPresentQueueIndex,                         // dstQueueFamilyIndex
            surface->mImages[backbuffer->mImageIndex],  // image
            {VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1}     // subresourceRange
    };

    mResetCommandBuffer(backbuffer->mTransitionCmdBuffers[1], 0);
    VkCommandBufferBeginInfo info;
    memset(&info, 0, sizeof(VkCommandBufferBeginInfo));
    info.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO;
    info.flags = 0;
    mBeginCommandBuffer(backbuffer->mTransitionCmdBuffers[1], &info);
    mCmdPipelineBarrier(backbuffer->mTransitionCmdBuffers[1], srcStageMask, dstStageMask, 0, 0,
                        nullptr, 0, nullptr, 1, &imageMemoryBarrier);
    mEndCommandBuffer(backbuffer->mTransitionCmdBuffers[1]);

    surface->mImageInfos[backbuffer->mImageIndex].mImageLayout = VK_IMAGE_LAYOUT_PRESENT_SRC_KHR;

    // insert the layout transfer into the queue and wait on the acquire
    VkSubmitInfo submitInfo;
    memset(&submitInfo, 0, sizeof(VkSubmitInfo));
    submitInfo.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO;
    submitInfo.waitSemaphoreCount = 0;
    submitInfo.pWaitDstStageMask = 0;
    submitInfo.commandBufferCount = 1;
    submitInfo.pCommandBuffers = &backbuffer->mTransitionCmdBuffers[1];
    submitInfo.signalSemaphoreCount = 1;
    // When this command buffer finishes we will signal this semaphore so that we know it is now
    // safe to present the image to the screen.
    submitInfo.pSignalSemaphores = &backbuffer->mRenderSemaphore;

    // Attach second fence to submission here so we can track when the command buffer finishes.
    mQueueSubmit(mBackendContext->fQueue, 1, &submitInfo, backbuffer->mUsageFences[1]);

    // Submit present operation to present queue. We use a semaphore here to make sure all rendering
    // to the image is complete and that the layout has been change to present on the graphics
    // queue.
    const VkPresentInfoKHR presentInfo = {
            VK_STRUCTURE_TYPE_PRESENT_INFO_KHR,  // sType
            NULL,                                // pNext
            1,                                   // waitSemaphoreCount
            &backbuffer->mRenderSemaphore,       // pWaitSemaphores
            1,                                   // swapchainCount
            &surface->mSwapchain,                // pSwapchains
            &backbuffer->mImageIndex,            // pImageIndices
            NULL                                 // pResults
    };

    mQueuePresentKHR(mPresentQueue, &presentInfo);

    surface->mBackbuffer.reset();
    surface->mImageInfos[backbuffer->mImageIndex].mLastUsed = surface->mCurrentTime;
    surface->mImageInfos[backbuffer->mImageIndex].mInvalid = false;
    surface->mCurrentTime++;
}

这些流程都和OpenGL是一样的,初始化,绑定Surface,绘制,提交,所以就不细说了,对Vulkan有兴趣的,可以深入的去研究。至此Android中的另一个图像生产者Vulkan生产图像的流程也讲完了。

结尾

OpenGL,Skia,Vulkan都是跨平台的图形生产者,我们不仅仅可以在Android设备上使用,我们也可以在IOS设备上使用,也可以在Windows设备上使用,使用的流程基本和上面一致,但是需要适配设备的原生窗口和缓冲,所以掌握了Android是如何绘制图像的,我们也具备了掌握其他任何设备上是如何绘制图像的能力。

在下一篇文章掌握Android图像显示原理(下),我会介绍Android图像渲染原理的最后一部分:图像缓冲区。这三部分如果都能掌握,我们基本就能掌握Android中图像绘制的原理了。