阅读 780

Transform和Task之间有关?| Gradle

之前有个朋友在面总部的安卓岗的说被问了一些技术问题,感觉还是很有难度,而且也很有水平。感觉在总部的开发还是都很强的啊。

  1. 自定义Transform和其他系统Transform执行的顺序
  2. Transform和Task的关系
  3. Transform是如何被执行的

什么是Task

其实核心还是要先从什么是Task讲起了。

一个Task代表一个构建工作的原子操作,例如编译calsses或者生成javadoc。 Gradle中,每一个待编译的工程都叫一个Project。每一个Project在构建的时候都包含一系列的Task。比如一个Android APK的编译可能包含:Java源码编译Task、资源编译Task、JNI编译Task、lint检查Task、打包生成APK的Task、签名Task等。插件本身就是包含了若干Task的。

简单的说我们的项目编译以assembleDebug为例子,会顺序执行非常多的gradle task任务,举个例子比如说aapt,javac,kotlinc等等,他们都是作为一个task存在的。

AGP中的Transform

   AppExtension appExtension = project.getExtensions().getByType(AppExtension.class);
   appExtension.registerTransform(new DoubleTabTransform(project));
复制代码

当我们在编写一个含有Transformplugin插件的时候,只是对安卓的AppExtension注册了一个Transform,那么Transform的本质到底是什么呢?

高能预警,下面的源代码比较长,可以考虑直接跳过看结论,但是看得懂的同学最好可以学习下。

public class LibraryTaskManager extends TaskManager {

    @Override
    public void createTasksForVariantScope(@NonNull final VariantScope variantScope) {
        ...
         // ----- External Transforms -----
        // apply all the external transforms.
        List<Transform> customTransforms = extension.getTransforms();
        List<List<Object>> customTransformsDependencies = extension.getTransformsDependencies();

        for (int i = 0, count = customTransforms.size(); i < count; i++) {
            Transform transform = customTransforms.get(i);

            // Check the transform only applies to supported scopes for libraries:
            // We cannot transform scopes that are not packaged in the library
            // itself.
            Sets.SetView<? super Scope> difference =
                    Sets.difference(transform.getScopes(), TransformManager.PROJECT_ONLY);
            if (!difference.isEmpty()) {
                String scopes = difference.toString();
                globalScope
                        .getAndroidBuilder()
                        .getIssueReporter()
                        .reportError(
                                Type.GENERIC,
                                new EvalIssueException(
                                        String.format(
                                                "Transforms with scopes '%s' cannot be applied to library projects.",
                                                scopes)));
            }

            List<Object> deps = customTransformsDependencies.get(i);
            transformManager.addTransform(
                    taskFactory,
                    variantScope,
                    transform,
                    null,
                    task -> {
                        if (!deps.isEmpty()) {
                            task.dependsOn(deps);
                        }
                    },
                    taskProvider -> {
                        // if the task is a no-op then we make assemble task
                        // depend on it.
                        if (transform.getScopes().isEmpty()) {
                            TaskFactoryUtils.dependsOn(
                                    variantScope.getTaskContainer().getAssembleTask(),
                                    taskProvider);
                        }
                    });
        }

        // Now add transforms for intermediate publishing (projects to projects).
        File jarOutputFolder = variantScope.getIntermediateJarOutputFolder();
        File mainClassJar = new File(jarOutputFolder, FN_CLASSES_JAR);
        File mainResJar = new File(jarOutputFolder, FN_INTERMEDIATE_RES_JAR);
        LibraryIntermediateJarsTransform intermediateTransform =
                new LibraryIntermediateJarsTransform(
                        mainClassJar,
                        mainResJar,
                        variantConfig::getPackageFromManifest,
                        extension.getPackageBuildConfig());
        excludeDataBindingClassesIfNecessary(variantScope, intermediateTransform);

        BuildArtifactsHolder artifacts = variantScope.getArtifacts();
        transformManager.addTransform(
                taskFactory,
                variantScope,
                intermediateTransform,
                taskName -> {
                    // publish the intermediate classes.jar
                    artifacts.appendArtifact(
                            InternalArtifactType.LIBRARY_CLASSES,
                            ImmutableList.of(mainClassJar),
                            taskName);
                    // publish the res jar
                    artifacts.appendArtifact(
                            InternalArtifactType.LIBRARY_JAVA_RES,
                            ImmutableList.of(mainResJar),
                            taskName);
                },
                null,
                null);

        taskFactory.register(new LibraryDexingTask.CreationAction(variantScope));

        // Create a jar with both classes and java resources.  This artifact is not
        // used by the Android application plugin and the task usually don't need to
        // be executed.  The artifact is useful for other Gradle users who needs the
        // 'jar' artifact as API dependency.
        taskFactory.register(new ZipMergingTask.CreationAction(variantScope));

        // now add a transform that will take all the native libs and package
        // them into an intermediary folder. This processes only the PROJECT
        // scope.
        final File intermediateJniLibsFolder = new File(jarOutputFolder, FD_JNI);

        LibraryJniLibsTransform intermediateJniTransform =
                new LibraryJniLibsTransform(
                        "intermediateJniLibs",
                        intermediateJniLibsFolder,
                        TransformManager.PROJECT_ONLY);
        transformManager.addTransform(
                taskFactory,
                variantScope,
                intermediateJniTransform,
                taskName -> {
                    // publish the jni folder as intermediate
                    variantScope
                            .getArtifacts()
                            .appendArtifact(
                                    InternalArtifactType.LIBRARY_JNI,
                                    ImmutableList.of(intermediateJniLibsFolder),
                                    taskName);
                },
                null,
                null);

        // Now go back to fill the pipeline with transforms used when
        // publishing the AAR

        // first merge the resources. This takes the PROJECT and LOCAL_DEPS
        // and merges them together.
        createMergeJavaResTransform(variantScope);

        // ----- Minify next -----
        maybeCreateJavaCodeShrinkerTransform(variantScope);
        maybeCreateResourcesShrinkerTransform(variantScope);

        // now add a transform that will take all the class/res and package them
        // into the main and secondary jar files that goes in the AAR.
        // This transform technically does not use its transform output, but that's
        // ok. We use the transform mechanism to get incremental data from
        // the streams.
        // This is used for building the AAR.

        File classesJar = variantScope.getAarClassesJar();
        File libsDirectory = variantScope.getAarLibsDirectory();

        LibraryAarJarsTransform transform =
                new LibraryAarJarsTransform(
                        classesJar,
                        libsDirectory,
                        artifacts.hasArtifact(InternalArtifactType.ANNOTATIONS_TYPEDEF_FILE)
                                ? artifacts.getFinalArtifactFiles(
                                        InternalArtifactType.ANNOTATIONS_TYPEDEF_FILE)
                                : null,
                        variantConfig::getPackageFromManifest,
                        extension.getPackageBuildConfig());

        excludeDataBindingClassesIfNecessary(variantScope, transform);

        transformManager.addTransform(
                taskFactory,
                variantScope,
                transform,
                taskName -> {
                    variantScope
                            .getArtifacts()
                            .appendArtifact(
                                    InternalArtifactType.AAR_MAIN_JAR,
                                    ImmutableList.of(classesJar),
                                    taskName);
                    variantScope
                            .getArtifacts()
                            .appendArtifact(
                                    InternalArtifactType.AAR_LIBS_DIRECTORY,
                                    ImmutableList.of(libsDirectory),
                                    taskName);
                },
                null,
                null);

        // now add a transform that will take all the native libs and package
        // them into the libs folder of the bundle. This processes both the PROJECT
        // and the LOCAL_PROJECT scopes
        final File jniLibsFolder =
                variantScope.getIntermediateDir(InternalArtifactType.LIBRARY_AND_LOCAL_JARS_JNI);
        LibraryJniLibsTransform jniTransform =
                new LibraryJniLibsTransform(
                        "syncJniLibs",
                        jniLibsFolder,
                        TransformManager.SCOPE_FULL_LIBRARY_WITH_LOCAL_JARS);
        transformManager.addTransform(
                taskFactory,
                variantScope,
                jniTransform,
                taskName ->
                        variantScope
                                .getArtifacts()
                                .appendArtifact(
                                        InternalArtifactType.LIBRARY_AND_LOCAL_JARS_JNI,
                                        ImmutableList.of(jniLibsFolder),
                                        taskName),
                null,
                null);

        createLintTasks(variantScope);
        createBundleTask(variantScope);
    }
}
复制代码

自定义Transform和其他系统Transform执行的顺序

而且上述方法我们可以看出,任务还是会根据DAG(有向无环图)的拓扑排序将Task顺序执行。其中会包含一些系统的Transform,其顺序有可能会被插入到自定义的Transform之前,而有一些则会被放置在所有的Tranform执行之后。比如LibraryJniLibsTransform

拓扑排序(Topological Order)是指,将一个有向无环图(Directed Acyclic Graph简称DAG)进行排序进而得到一个有序的线性序列。

例如,一个项目包括A、B、C、D四个子部分来完成,并且A依赖于B和D,C依赖于D。现在要制定一个计划,写出A、B、C、D的执行顺序。这时,就可以利用到拓扑排序,它就是用来确定事物发生的顺序的。在拓扑排序中,如果存在一条从顶点A到顶点B的路径,那么在排序结果中B出现在A的后面。

Transform和Task的关系

从这部分源代码其实我们就可以看出,我们注册到AppExtension里面的Transform,之后所有的Transform会createTasksForVariantScope方法调用到,然后会把taskManger的addTransform方法。

@NonNull
 public <T extends Transform> Optional<TaskProvider<TransformTask>> addTransform(
         @NonNull TaskFactory taskFactory,
         @NonNull TransformVariantScope scope,
         @NonNull T transform,
         @Nullable PreConfigAction preConfigAction,
         @Nullable TaskConfigAction<TransformTask> configAction,
         @Nullable TaskProviderCallback<TransformTask> providerCallback) {

     if (!validateTransform(transform)) {
         // validate either throws an exception, or records the problem during sync
         // so it's safe to just return null here.
         return Optional.empty();
     }

     List<TransformStream> inputStreams = Lists.newArrayList();
     String taskName = scope.getTaskName(getTaskNamePrefix(transform));

     // get referenced-only streams
     List<TransformStream> referencedStreams = grabReferencedStreams(transform);

     // find input streams, and compute output streams for the transform.
     IntermediateStream outputStream = findTransformStreams(
             transform,
             scope,
             inputStreams,
             taskName,
             scope.getGlobalScope().getBuildDir());

     if (inputStreams.isEmpty() && referencedStreams.isEmpty()) {
         // didn't find any match. Means there is a broken order somewhere in the streams.
         issueReporter.reportError(
                 Type.GENERIC,
                 new EvalIssueException(
                         String.format(
                                 "Unable to add Transform '%s' on variant '%s': requested streams not available: %s+%s / %s",
                                 transform.getName(),
                                 scope.getFullVariantName(),
                                 transform.getScopes(),
                                 transform.getReferencedScopes(),
                                 transform.getInputTypes())));
         return Optional.empty();
     }

     //noinspection PointlessBooleanExpression
     if (DEBUG && logger.isEnabled(LogLevel.DEBUG)) {
         logger.debug("ADDED TRANSFORM(" + scope.getFullVariantName() + "):");
         logger.debug("\tName: " + transform.getName());
         logger.debug("\tTask: " + taskName);
         for (TransformStream sd : inputStreams) {
             logger.debug("\tInputStream: " + sd);
         }
         for (TransformStream sd : referencedStreams) {
             logger.debug("\tRef'edStream: " + sd);
         }
         if (outputStream != null) {
             logger.debug("\tOutputStream: " + outputStream);
         }
     }

     transforms.add(transform);

     // create the task...
     return Optional.of(
             taskFactory.register(
                     new TransformTask.CreationAction<>(
                             scope.getFullVariantName(),
                             taskName,
                             transform,
                             inputStreams,
                             referencedStreams,
                             outputStream,
                             recorder),
                     preConfigAction,
                     configAction,
                     providerCallback));
 }
复制代码

然后我们看transformManager.addTransform,直接观察最后一行代码,发现我们会往taskFactory中直接注册一个TransformTask,那么回答第一个问题,一个Transform就是gradle plugin中的一个原子的Task,所以Transform==Task

Transform是如何被执行的

当我们知道Transform的本质是Task之后就知道了,他会和一个普通的Task一样被执行,而且会按照特定的依赖顺序执行,而每个Task又是一个Gradle Project中的最基础的原子操作,所以Transform的执行顺序就是Task的执行顺序。

玩转Task

之前在写资源混淆编译速度优化的时候有些小心得分享给大家。

首先我们构造一个Task之后要先找到这个Task的前置任务依赖,然后我们可以把实际的Task内容执行在线程池内,充分的利用多线程的优势去构造一个项目,然后我们需要做的就是在后置的依赖任务执行之前,获取到任务的Task,之后在Task 的doFirst方法做一个await操作,这样就可以把充分利用cpu和多线程的优势,去优化一个Task任务了。

val manager = scope.transformManager
           val field = manager.javaClass.getDeclaredField("transforms").apply {
               isAccessible = true
           }
           val list = field.get(manager) as List<Transform>
           list.forEach {
               if (it is ShrinkResourcesTransform) {
                   val taskName = scope.getTaskName(getTaskNamePrefix(it))
                   val task = project.tasks.getByName(taskName)
                   task.doFirst {
                       val testTask = project.tasks.getByName(resGuardTaskName)
                               as TestTask
                       val taskLogger = TimeUsageHelper()
                       taskLogger.info("AwaitTask start ")
                       resProguardTask.await()
                       taskLogger.logTotalInfo("AwaitTask finish ")
                       Log.writeToFile(project)
                   }
                   task.dependsOn(testTask)
               }
           }
复制代码

上述代码就是先获取一个系统的Transform任务,然后在任务之前await上一个任务完成。

总结

其实这三个题目还是很有意思的,看看源代码就可以对进行一部分剖析,而且其实如果你仔细阅读的话其实可以在上述源代码中发现一些有意思的Task,比如mergeJavaResource(合并资源),ShrinkResourcesTransform(删除无用的资源)等等。

我的博客即将搬运同步至腾讯云+社区,邀请大家一同入驻:cloud.tencent.com/developer/s…