注册中心 Eureka 源码解析 —— Eureka-Client 初始化(三)之 EurekaClient

2,447 阅读10分钟
《Dubbo 实现原理与源码解析 —— 精品合集》 《Netty 实现原理与源码解析 —— 精品合集》
《Spring 实现原理与源码解析 —— 精品合集》 《MyBatis 实现原理与源码解析 —— 精品合集》
《Spring MVC 实现原理与源码解析 —— 精品合集》 《数据库实体设计合集》

摘要: 原创出处 www.iocoder.cn/Eureka/eure… 「芋道源码」欢迎转载,保留摘要,谢谢!

本文主要基于 Eureka 1.8.X 版本


🙂🙂🙂关注微信公众号:【芋道源码】有福利:

  1. RocketMQ / MyCAT / Sharding-JDBC 所有源码分析文章列表
  2. RocketMQ / MyCAT / Sharding-JDBC 中文注释源码 GitHub 地址
  3. 您对于源码的疑问每条留言将得到认真回复。甚至不知道如何读源码也可以请教噢
  4. 新的源码解析文章实时收到通知。每周更新一篇左右
  5. 认真的源码交流微信群。

1. 概述

本文接《Eureka 源码解析 —— Eureka-Client 初始化(二)之 EurekaClientConfig》,主要分享 Eureka-Client 自身初始化的过程的第三部分 —— EurekaClient,不包含 Eureka-Client 向 Eureka-Server 的注册过程( 🙂后面会另外文章分享 )。

Eureka-Client 自身初始化过程中,涉及到主要对象如下图:

  1. 创建 EurekaInstanceConfig对象
  2. 使用 EurekaInstanceConfig对象 创建 InstanceInfo对象
  3. 使用 EurekaInstanceConfig对象 + InstanceInfo对象 创建 ApplicationInfoManager对象
  4. 创建 EurekaClientConfig对象
  5. 使用 ApplicationInfoManager对象 + EurekaClientConfig对象 创建 EurekaClient对象

考虑到整个初始化的过程中涉及的配置特别多,拆分成三篇文章:

  1. (一)EurekaInstanceConfig)
  2. (二)EurekaClientConfig
  3. 【本文】(三)EurekaClient

下面我们来看看每个的实现。

推荐 Spring Cloud 书籍

推荐 Spring Cloud 视频

2. EurekaClient

com.netflix.discovery.EurekaClient,Eureka-Client 接口,声明如下方法:

  • 提供多种方法获取应用集合(com.netflix.discovery.shared.Applications) 和 应用实例信息集合( com.netflix.appinfo.InstanceInfo )。
  • 提供方法获取本地客户端信息,例如,应用管理器( com.netflix.appinfo.ApplicationInfoManager )和 Eureka-Client 配置( com.netflix.discovery.EurekaClientConfig )。
  • 提供方法注册本地客户端的健康检查和 Eureka 事件监听器。

另外,Eureka 2.X 版本正在开发,该接口为 Eureka 1.X 和 2.X 提供平滑过渡接口。

This interface does NOT try to clean up the current client interface for eureka 1.x. Rather it tries to provide an easier transition path from eureka 1.x to eureka 2.x.

2.1 LookupService

com.netflix.discovery.shared.LookupService,查找服务接口,提供简单单一的方式获取应用集合(com.netflix.discovery.shared.Applications) 和 应用实例信息集合( com.netflix.appinfo.InstanceInfo )。

  • 在 Eureka-Client 里,EurekaClient 继承该接口。
  • 在 Eureka-Server 里,com.netflix.eureka.registry.InstanceRegistry 继承该接口。

3. DiscoveryClient

com.netflix.discovery.DiscoveryClient,实现 EurekaClient 接口,用于与 Eureka-Server 交互。实现如下方法:

  • 向 Eureka-Server 注册自身服务
  • 向 Eureka-Server 续约自身服务
  • 向 Eureka-Server 取消自身服务,当关闭时
  • 从 Eureka-Server 查询应用集合和应用实例信息
  • 简单来理解,对 Eureka-Server 服务的增删改查

3.1 构造方法参数

DiscoveryClient 完整构造方法需要传入四个参数,实现代码如下:

DiscoveryClient(ApplicationInfoManager applicationInfoManager, EurekaClientConfig config, AbstractDiscoveryClientOptionalArgs args,
                    Provider<BackupRegistry> backupRegistryProvider) {
     // .... 省略代码
}
  • ApplicationInfoManager,在《Eureka 源码解析 —— Eureka-Client 初始化(一)之 EurekaInstanceConfig》有详细解析。
  • EurekaClientConfig,在《Eureka 源码解析 —— Eureka-Client 初始化(二)之 EurekaClientConfig》有详细解析。
  • com.netflix.discovery.BackupRegistry,备份注册中心接口。当 Eureka-Client 启动时,无法从 Eureka-Server 读取注册信息(可能挂了),从备份注册中心读取注册信息。实现代码如下:

    // BackupRegistry.java
    public interface BackupRegistry {
    
        Applications fetchRegistry();
    
        Applications fetchRegistry(String[] includeRemoteRegions);
    }
    
    // NotImplementedRegistryImpl.java
    public class NotImplementedRegistryImpl implements BackupRegistry {
    
        @Override
        public Applications fetchRegistry() {
            return null;
        }
    
        @Override
        public Applications fetchRegistry(String[] includeRemoteRegions) {
            return null;
        }
    }
    
    • com.netflix.discovery.NotImplementedRegistryImpl 可以看出,目前 Eureka-Client 未提供合适的默认实现。
  • com.netflix.discovery.AbstractDiscoveryClientOptionalArgs,DiscoveryClient 可选参数抽象基类。不同于上面三个必填参数,该参数是选填参数,实际生产下使用较少。实现代码如下:

    public abstract class AbstractDiscoveryClientOptionalArgs<T> {
    
        
        Provider<HealthCheckCallback> healthCheckCallbackProvider;
        
        Provider<HealthCheckHandler> healthCheckHandlerProvider;
        
        PreRegistrationHandler preRegistrationHandler;
        
        Collection<T> additionalFilters;
        
        EurekaJerseyClient eurekaJerseyClient;
        
        TransportClientFactories transportClientFactories;
        
        private Set<EurekaEventListener> eventListeners;
    }
    
    • com.netflix.appinfo.HealthCheckCallback,健康检查回调接口,目前已经废弃,使用 HealthCheckHandler 替代,你可以不关注该参数
    • com.netflix.appinfo.HealthCheckHandler,健康检查处理器接口,目前暂未提供合适的默认实现,唯一提供的 com.netflix.appinfo.HealthCheckCallbackToHandlerBridge,用于将 HealthCheckCallback 桥接成 HealthCheckHandler,实现代码如下:

      // HealthCheckHandler.java
      public interface HealthCheckHandler {
          
          InstanceInfo.InstanceStatus getStatus(InstanceInfo.InstanceStatus currentStatus);
      
      }
      
      // HealthCheckCallbackToHandlerBridge.java
      public class HealthCheckCallbackToHandlerBridge implements HealthCheckHandler {
      
          private final HealthCheckCallback callback;
      
          public HealthCheckCallbackToHandlerBridge() {
              callback = null;
          }
      
          public HealthCheckCallbackToHandlerBridge(HealthCheckCallback callback) {
              this.callback = callback;
          }
      
          @Override
          public InstanceInfo.InstanceStatus getStatus(InstanceInfo.InstanceStatus currentStatus) {
              if (null == callback || InstanceInfo.InstanceStatus.STARTING == currentStatus
                      || InstanceInfo.InstanceStatus.OUT_OF_SERVICE == currentStatus) { // Do not go to healthcheck handler if the status is starting or OOS.
                  return currentStatus;
              }
      
              return callback.isHealthy() ? InstanceInfo.InstanceStatus.UP : InstanceInfo.InstanceStatus.DOWN;
          }
      
      }
      
    • com.netflix.discovery.PreRegistrationHandler,向 Eureka-Server 注册之前的处理器接口,目前暂未提供默认实现。通过实现该接口,可以在注册前做一些自定义的处理。实现代码如下:

      public interface PreRegistrationHandler {
          
          void beforeRegistration();
      
      }
      
      • x
      • additionalFilters,Jersey 过滤器集合。这里声明泛型 <T> 的原因,Jersey 1.X 和 Jersey 2.X 的过滤器接口不同,通过泛型来支持。实现代码如下:

        // Jersey1DiscoveryClientOptionalArgs.java
        public class Jersey1DiscoveryClientOptionalArgs extends AbstractDiscoveryClientOptionalArgs<ClientFilter> {
         }
         
        // Jersey2DiscoveryClientOptionalArgs.java
        public class Jersey2DiscoveryClientOptionalArgs extends AbstractDiscoveryClientOptionalArgs<ClientRequestFilter> {
         }
         
        // DiscoveryClientOptionalArgs.java
        public static class DiscoveryClientOptionalArgs extends Jersey1DiscoveryClientOptionalArgs {
        
        }
        
        • Jersey 1.X 使用 ClientFilter 。ClientFilter 目前有两个过滤器实现:EurekaIdentityHeaderFilter 、DynamicGZIPContentEncodingFilter 。
        • Jersey 2.X 使用 ClientRequestFilter 。
        • DiscoveryClient 使用 DiscoveryClientOptionalArgs,即 Jersey 1.X 。
    • eurekaJerseyClient,Jersey 客户端。该参数目前废弃,使用下面 TransportClientFactories 参数来进行生成。
    • com.netflix.discovery.shared.transport.jersey.TransportClientFactories,生成 Jersey 客户端工厂的工厂接口。目前有 Jersey1TransportClientFactories 、Jersey2TransportClientFactories 两个实现。TransportClientFactories 实现代码如下:

      // TransportClientFactories.java
      public interface TransportClientFactories<F> {
        
            @Deprecated
            TransportClientFactory newTransportClientFactory(final Collection<F> additionalFilters,
                                                             final EurekaJerseyClient providedJerseyClient);
        
            TransportClientFactory newTransportClientFactory(final EurekaClientConfig clientConfig,
                                                             final Collection<F> additionalFilters,
                                                             final InstanceInfo myInstanceInfo);
      }
      
      // TransportClientFactory.java
      public interface TransportClientFactory {
      
            EurekaHttpClient newClient(EurekaEndpoint serviceUrl);
          
            void shutdown();
          
      }
      
      • 第一个方法已经废弃,这就是为什么说上面的 eurekaJerseyClient 参数( 不是 EurekaJerseyClient 类)已经废弃,被第二个方法取代。相比来说,第二个方法对 EurekaJerseyClient 创建封装会更好。
    • com.netflix.discovery.EurekaEventListener,Eureka 事件监听器。实现代码如下:

      // EurekaEventListener.java
      public interface EurekaEventListener {
      }
      
      // EurekaEvent.java
      public interface EurekaEvent {
      }
      
      // DiscoveryEvent.java
      public abstract class DiscoveryEvent implements EurekaEvent {
      
             private final long timestamp;
      
      }
      

3.2 构造方法

DiscoveryClient 的构造方法实现代码相对较多,已经将代码切块 + 中文注册,点击 DiscoveryClient 链接,对照下面每个小结阅读理解。

3.2.1 赋值 AbstractDiscoveryClientOptionalArgs

// DiscoveryClient.java 构造方法
if (args != null) {
  this.healthCheckHandlerProvider = args.healthCheckHandlerProvider;
  this.healthCheckCallbackProvider = args.healthCheckCallbackProvider;
  this.eventListeners.addAll(args.getEventListeners());
  this.preRegistrationHandler = args.preRegistrationHandler;
} else {
  this.healthCheckCallbackProvider = null;
  this.healthCheckHandlerProvider = null;
  this.preRegistrationHandler = null;
}

3.2.2 赋值 ApplicationInfoManager、EurekaClientConfig

// DiscoveryClient.java 构造方法
this.applicationInfoManager = applicationInfoManager;
InstanceInfo myInfo = applicationInfoManager.getInfo();

clientConfig = config;
staticClientConfig = clientConfig;
transportConfig = config.getTransportConfig();
instanceInfo = myInfo;
if (myInfo != null) {
  appPathIdentifier = instanceInfo.getAppName() + "/" + instanceInfo.getId(); // 无实际业务用途,用于打 logger
} else {
  logger.warn("Setting instanceInfo to a passed in null value");
}

3.2.3 赋值 BackupRegistry

this.backupRegistryProvider = backupRegistryProvider;

3.2.4 初始化 InstanceInfoBasedUrlRandomizer

TODO[0016]:InstanceInfoBasedUrlRandomizer

this.urlRandomizer = new EndpointUtils.InstanceInfoBasedUrlRandomizer(instanceInfo);

3.2.5 初始化 Applications 在本地的缓存

// DiscoveryClient.java 变量

private final AtomicReference<Applications> localRegionApps = new AtomicReference<Applications>();

private final AtomicLong fetchRegistryGeneration;

// DiscoveryClient.java 构造方法
localRegionApps.set(new Applications());

fetchRegistryGeneration = new AtomicLong(0);

3.2.6 获取哪些 Region 集合的注册信息

// DiscoveryClient.java 变量

private final AtomicReference<String> remoteRegionsToFetch;

private final AtomicReference<String[]> remoteRegionsRef;

// DiscoveryClient.java 构造方法
remoteRegionsToFetch = new AtomicReference<>(clientConfig.fetchRegistryForRemoteRegions());
remoteRegionsRef = new AtomicReference<>(remoteRegionsToFetch.get() == null ? null : remoteRegionsToFetch.get().split(","));

3.2.7 初始化拉取、心跳的监控

// DiscoveryClient.java 变量

private volatile long lastSuccessfulRegistryFetchTimestamp = -1;

private volatile long lastSuccessfulHeartbeatTimestamp = -1;

private final ThresholdLevelsMetric heartbeatStalenessMonitor;

private final ThresholdLevelsMetric registryStalenessMonitor;

// DiscoveryClient.java 构造方法
if (config.shouldFetchRegistry()) {
  this.registryStalenessMonitor = new ThresholdLevelsMetric(this, METRIC_REGISTRY_PREFIX + "lastUpdateSec_", new long[]{15L, 30L, 60L, 120L, 240L, 480L});
} else {
  this.registryStalenessMonitor = ThresholdLevelsMetric.NO_OP_METRIC;
}

if (config.shouldRegisterWithEureka()) {
  this.heartbeatStalenessMonitor = new ThresholdLevelsMetric(this, METRIC_REGISTRATION_PREFIX + "lastHeartbeatSec_", new long[]{15L, 30L, 60L, 120L, 240L, 480L});
} else {
  this.heartbeatStalenessMonitor = ThresholdLevelsMetric.NO_OP_METRIC;
}
  • 每次成功向 Eureka-Serve 心跳或者从从 Eureka-Server 拉取注册信息后,都会更新相应时间戳。
  • 配合 Netflix Servo 实现监控信息采集。
  • com.netflix.discovery.util.ThresholdLevelsMetric 感兴趣的同学可以点击链接查看。本文暂不拓展开,后面另开文章分享。(TODO[0012]:监控相关)

3.2.8 结束初始化,当无需和 Eureka-Server 交互

// DiscoveryClient.java 构造方法
if (!config.shouldRegisterWithEureka() && !config.shouldFetchRegistry()) {
  logger.info("Client configured to neither register nor query for data.");
  scheduler = null;
  heartbeatExecutor = null;
  cacheRefreshExecutor = null;
  eurekaTransport = null;
  instanceRegionChecker = new InstanceRegionChecker(new PropertyBasedAzToRegionMapper(config), clientConfig.getRegion());

  // This is a bit of hack to allow for existing code using DiscoveryManager.getInstance()
  // to work with DI'd DiscoveryClient
  DiscoveryManager.getInstance().setDiscoveryClient(this);
  DiscoveryManager.getInstance().setEurekaClientConfig(config);

  initTimestampMs = System.currentTimeMillis();
  logger.info("Discovery Client initialized at timestamp {} with initial instances count: {}",
          initTimestampMs, this.getApplications().size());

  return;  // no need to setup up an network tasks and we are done
}

3.2.9 初始化线程池

// DiscoveryClient.java 变量

private final ScheduledExecutorService scheduler;
// additional executors for supervised subtasks

private final ThreadPoolExecutor heartbeatExecutor;

private final ThreadPoolExecutor cacheRefreshExecutor;

// DiscoveryClient.java 构造方法
// default size of 2 - 1 each for heartbeat and cacheRefresh
scheduler = Executors.newScheduledThreadPool(2,
     new ThreadFactoryBuilder()
             .setNameFormat("DiscoveryClient-%d")
             .setDaemon(true)
             .build());

heartbeatExecutor = new ThreadPoolExecutor(
     1, clientConfig.getHeartbeatExecutorThreadPoolSize(), 0, TimeUnit.SECONDS,
     new SynchronousQueue<Runnable>(),
     new ThreadFactoryBuilder()
             .setNameFormat("DiscoveryClient-HeartbeatExecutor-%d")
             .setDaemon(true)
             .build()
);  // use direct handoff

cacheRefreshExecutor = new ThreadPoolExecutor(
     1, clientConfig.getCacheRefreshExecutorThreadPoolSize(), 0, TimeUnit.SECONDS,
     new SynchronousQueue<Runnable>(),
     new ThreadFactoryBuilder()
             .setNameFormat("DiscoveryClient-CacheRefreshExecutor-%d")
             .setDaemon(true)
             .build()
);  // use direct handoff
  • scheduler定时任务线程池,初始化大小为 2,一个给 heartbeatExecutor,一个给 cacheRefreshExecutor
  • heartbeatExecutorcacheRefreshExecutor 在提交给 scheduler 才声明具体的任务

3.2.10 初始化 Eureka 网络通信相关

// DiscoveryClient.java 构造方法
eurekaTransport = new EurekaTransport();
scheduleServerEndpointTask(eurekaTransport, args);

3.2.11 初始化 InstanceRegionChecker

// DiscoveryClient.java 构造方法
AzToRegionMapper azToRegionMapper;
if (clientConfig.shouldUseDnsForFetchingServiceUrls()) {
    azToRegionMapper = new DNSBasedAzToRegionMapper(clientConfig);
} else {
    azToRegionMapper = new PropertyBasedAzToRegionMapper(clientConfig);
}
if (null != remoteRegionsToFetch.get()) {
    azToRegionMapper.setRegionsToFetch(remoteRegionsToFetch.get().split(","));
}
instanceRegionChecker = new InstanceRegionChecker(azToRegionMapper, clientConfig.getRegion());
  • com.netflix.discovery.AzToRegionMapper,主要用于亚马逊 AWS,跳过。
  • com.netflix.discovery.InstanceRegionChecker,应用实例信息区域( region )校验,实现代码如下:

    public class InstanceRegionChecker {
    
        // ... 省略和亚马逊 AWS 相关的属性和方法
    
        
        private final String localRegion;
    
        public boolean isLocalRegion(@Nullable String instanceRegion) {
            return null == instanceRegion || instanceRegion.equals(localRegion); // no region == local
        }
    
        public String getLocalRegion() {
            return localRegion;
        }
    
    }
    

3.2.12 从 Eureka-Server 拉取注册信息

// DiscoveryClient.java 构造方法
if (clientConfig.shouldFetchRegistry() && !fetchRegistry(false)) {
  fetchRegistryFromBackup();
}
  • 调用 #fetchRegistry(false) 方法,从 Eureka-Server 初始拉取注册信息。在(TO后文链接)详细解析。
  • 调用 #fetchRegistryFromBackup() 方法,若初始拉取注册信息失败,从备份注册中心获取。实现代码如下:

    // DiscoveryClient.java
    private void fetchRegistryFromBackup() {
        try {
            @SuppressWarnings("deprecation")
            BackupRegistry backupRegistryInstance = newBackupRegistryInstance();
            if (null == backupRegistryInstance) { // backward compatibility with the old protected method, in case it is being used.
                backupRegistryInstance = backupRegistryProvider.get();
            }
            if (null != backupRegistryInstance) {
                Applications apps = null;
                if (isFetchingRemoteRegionRegistries()) {
                    String remoteRegionsStr = remoteRegionsToFetch.get();
                    if (null != remoteRegionsStr) {
                        apps = backupRegistryInstance.fetchRegistry(remoteRegionsStr.split(","));
                    }
                } else {
                    apps = backupRegistryInstance.fetchRegistry();
                }
                if (apps != null) {
                    final Applications applications = this.filterAndShuffle(apps);
                    applications.setAppsHashCode(applications.getReconcileHashCode());
                    localRegionApps.set(applications);
                    logTotalInstances();
                    logger.info("Fetched registry successfully from the backup");
                }
            } else {
                logger.warn("No backup registry instance defined & unable to find any discovery servers.");
            }
        } catch (Throwable e) {
            logger.warn("Cannot fetch applications from apps although backup registry was specified", e);
        }
    }
    
    • BackupRegistry 目前暂未提供默认实现,需要自行相关逻辑。

3.2.13 执行向 Eureka-Server 注册之前的处理器

// DiscoveryClient.java 构造方法
// call and execute the pre registration handler before all background tasks (inc registration) is started
if (this.preRegistrationHandler != null) {
  this.preRegistrationHandler.beforeRegistration();
}

3.2.14 初始化定时任务

// DiscoveryClient.java 构造方法
initScheduledTasks();

// DiscoveryClient.java
private void initScheduledTasks() {
   // 从 Eureka-Server 拉取注册信息执行器
   if (clientConfig.shouldFetchRegistry()) {
       // registry cache refresh timer
       int registryFetchIntervalSeconds = clientConfig.getRegistryFetchIntervalSeconds();
       int expBackOffBound = clientConfig.getCacheRefreshExecutorExponentialBackOffBound();
       scheduler.schedule(
               new TimedSupervisorTask(
                       "cacheRefresh",
                       scheduler,
                       cacheRefreshExecutor,
                       registryFetchIntervalSeconds,
                       TimeUnit.SECONDS,
                       expBackOffBound,
                       new CacheRefreshThread()
               ),
               registryFetchIntervalSeconds, TimeUnit.SECONDS);
   }

   // 向 Eureka-Server 心跳(续租)执行器
   if (clientConfig.shouldRegisterWithEureka()) {
       int renewalIntervalInSecs = instanceInfo.getLeaseInfo().getRenewalIntervalInSecs();
       int expBackOffBound = clientConfig.getHeartbeatExecutorExponentialBackOffBound();
       logger.info("Starting heartbeat executor: " + "renew interval is: " + renewalIntervalInSecs);

       // Heartbeat timer
       scheduler.schedule(
               new TimedSupervisorTask(
                       "heartbeat",
                       scheduler,
                       heartbeatExecutor,
                       renewalIntervalInSecs,
                       TimeUnit.SECONDS,
                       expBackOffBound,
                       new HeartbeatThread()
               ),
               renewalIntervalInSecs, TimeUnit.SECONDS);

       // InstanceInfo replicator
       instanceInfoReplicator = new InstanceInfoReplicator(
               this,
               instanceInfo,
               clientConfig.getInstanceInfoReplicationIntervalSeconds(),
               2); // burstSize

       statusChangeListener = new ApplicationInfoManager.StatusChangeListener() {
           @Override
           public String getId() {
               return "statusChangeListener";
           }

           @Override
           public void notify(StatusChangeEvent statusChangeEvent) {
               if (InstanceStatus.DOWN == statusChangeEvent.getStatus() ||
                       InstanceStatus.DOWN == statusChangeEvent.getPreviousStatus()) {
                   // log at warn level if DOWN was involved
                   logger.warn("Saw local status change event {}", statusChangeEvent);
               } else {
                   logger.info("Saw local status change event {}", statusChangeEvent);
               }
               instanceInfoReplicator.onDemandUpdate();
           }
       };

       if (clientConfig.shouldOnDemandUpdateStatusChange()) {
           applicationInfoManager.registerStatusChangeListener(statusChangeListener);
       }

       instanceInfoReplicator.start(clientConfig.getInitialInstanceInfoReplicationIntervalSeconds());
   } else {
       logger.info("Not registering with Eureka server per configuration");
   }
}

3.2.15 向 Servo 注册监控

// DiscoveryClient.java 构造方法
try {
  Monitors.registerObject(this);
} catch (Throwable e) {
  logger.warn("Cannot register timers", e);
}

3.2.16 初始化完成

// DiscoveryClient.java 变量

private final long initTimestampMs;

// DiscoveryClient.java 构造方法
// 【3.2.16】初始化完成
// This is a bit of hack to allow for existing code using DiscoveryManager.getInstance()
// to work with DI'd DiscoveryClient
DiscoveryManager.getInstance().setDiscoveryClient(this);
DiscoveryManager.getInstance().setEurekaClientConfig(config);

initTimestampMs = System.currentTimeMillis();
logger.info("Discovery Client initialized at timestamp {} with initial instances count: {}",
      initTimestampMs, this.getApplications().size());

666. 彩蛋

知识星球

由于笔者是边理解源码边输出博客内容,如果有错误或者不清晰的地方,欢迎微笑给我的微信公众号( 芋道源码 ) 留言,我会仔细回复。感谢 + 1024。

后面文章不断更新,会慢慢完善本文中的。

推荐参考阅读: