摘要: 原创出处 http://www.iocoder.cn/Apollo/service-register-discovery/ 「芋道源码」欢迎转载,保留摘要,谢谢!
- 1. 概述
- 2. Eureka Server
- 3. Meta Service
- 4. ConfigServiceLocator
- 5. AdminServiceAddressLocator
- 666. 彩蛋
1. 概述
老艿艿:本系列假定胖友已经阅读过 《Apollo 官方 wiki 文档》 ,特别是 《架构模块》 。
本文分享 Apollo 服务的注册与发现。如下图所示:
- 各服务的介绍,见 《各模块概要介绍》 。
- 默认情况下,Config Service、Meta Service、Eureka Server 统一部署在 Config Service 中。如果想要使用单独的 Eureka Server,参见 《将 Config Service 和 Admin Service 注册到单独的 Eureka Server 上》 。
2. Eureka Server
2.1 启动 Eureka Server
在 apollo-configservice
项目中,com.ctrip.framework.apollo.configservice.ConfigServiceApplication
中,通过 @EnableEurekaServer
注解启动 Eureka Server 。代码如下:
@EnableEurekaServer // 启动 Eureka Server
@EnableAspectJAutoProxy
@EnableAutoConfiguration // (exclude = EurekaClientConfigBean.class)
@Configuration
@EnableTransactionManagement
@PropertySource(value = {"classpath:configservice.properties"})
@ComponentScan(basePackageClasses = {ApolloCommonConfig.class,
ApolloBizConfig.class,
ConfigServiceApplication.class,
ApolloMetaServiceConfig.class})
public class ConfigServiceApplication {
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext context = new SpringApplicationBuilder(ConfigServiceApplication.class).run(args);
context.addApplicationListener(new ApplicationPidFileWriter());
context.addApplicationListener(new EmbeddedServerPortFileWriter());
}
}
-
第一行的
@EnableEurekaServer
注解,启动 Eureka Server 。基于 Spring Cloud Eureka ,需要在 Maven 的pom.xml
中申明如下依赖:<!-- eureka --> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-eureka-server</artifactId> <exclusions> <exclusion> <artifactId> spring-cloud-starter-archaius </artifactId> <groupId>org.springframework.cloud</groupId> </exclusion> <exclusion> <artifactId>spring-cloud-starter-ribbon</artifactId> <groupId>org.springframework.cloud</groupId> </exclusion> <exclusion> <artifactId>ribbon-eureka</artifactId> <groupId>com.netflix.ribbon</groupId> </exclusion> <exclusion> <artifactId>aws-java-sdk-core</artifactId> <groupId>com.amazonaws</groupId> </exclusion> <exclusion> <artifactId>aws-java-sdk-ec2</artifactId> <groupId>com.amazonaws</groupId> </exclusion> <exclusion> <artifactId>aws-java-sdk-autoscaling</artifactId> <groupId>com.amazonaws</groupId> </exclusion> <exclusion> <artifactId>aws-java-sdk-sts</artifactId> <groupId>com.amazonaws</groupId> </exclusion> <exclusion> <artifactId>aws-java-sdk-route53</artifactId> <groupId>com.amazonaws</groupId> </exclusion> <!-- duplicated with spring-security-core --> <exclusion> <groupId>org.springframework.security</groupId> <artifactId>spring-security-crypto</artifactId> </exclusion> </exclusions> </dependency> <!-- end of eureka -->
那么 Eureka Server 怎么构建成集群呢?答案在 「2.2 注册到 Eureka Client」 中。
2.2 注册到 Eureka Client
在 apollo-biz
项目中,com.ctrip.framework.apollo.biz.eureka.ApolloEurekaClientConfig
中,声明 Eureka 的配置。代码如下:
@Component
@Primary
public class ApolloEurekaClientConfig extends EurekaClientConfigBean {
@Autowired
private BizConfig bizConfig;
/**
* Assert only one zone: defaultZone, but multiple environments.
*/
@Override
public List<String> getEurekaServerServiceUrls(String myZone) {
List<String> urls = bizConfig.eurekaServiceUrls();
return CollectionUtils.isEmpty(urls) ? super.getEurekaServerServiceUrls(myZone) : urls;
}
@Override
public boolean equals(Object o) {
return super.equals(o);
}
}
-
@Primary
注解,保证优先级。 -
#getEurekaServerServiceUrls(myZone)
方法,调用BizConfig#eurekaServiceUrls()
方法,从 ServerConfig 的"eureka.service.url"
配置项,获得 Eureka Server 地址。代码如下:// 获得 Eureka 服务器地址的数组 public List<String> eurekaServiceUrls() { // 获得配置值 String configuration = getValue("eureka.service.url", ""); // 分隔成 List if (Strings.isNullOrEmpty(configuration)) { return Collections.emptyList(); } return splitter.splitToList(configuration); }
- Eureka Server 共享该配置,从而形成 Eureka Server 集群。
-
基于 Spring Cloud Eureka ,需要在 Maven 的
pom.xml
中申明如下依赖:<!-- eureka --> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-eureka</artifactId> </dependency> <!-- end of eureka -->
apollo-adminservice
和 apollo-configservice
项目,引入 apollo-biz
项目,启动 Eureka Client ,向 Eureka Server 注册自己为实例。通过 .properties
配置实例名:
// FROM adminservice.properties
spring.application.name= apollo-adminservice
// FROM configservice.properties
spring.application.name= apollo-configservice
3. Meta Service
在 apollo-configservice
项目中,metaservice
包下,看到所有 Meta Service 的类,如下图:
3.1 ApolloMetaServiceConfig
@EnableAutoConfiguration
@Configuration
@ComponentScan(basePackageClasses = ApolloMetaServiceConfig.class)
public class ApolloMetaServiceConfig {
}
3.2 ServiceController
@RestController
@RequestMapping("/services")
public class ServiceController {
@Autowired
private DiscoveryService discoveryService;
@RequestMapping("/meta")
public List<ServiceDTO> getMetaService() {
List<InstanceInfo> instances = discoveryService.getMetaServiceInstances();
List<ServiceDTO> result = instances.stream().map(new Function<InstanceInfo, ServiceDTO>() {
@Override
public ServiceDTO apply(InstanceInfo instance) {
ServiceDTO service = new ServiceDTO();
service.setAppName(instance.getAppName());
service.setInstanceId(instance.getInstanceId());
service.setHomepageUrl(instance.getHomePageUrl());
return service;
}
}).collect(Collectors.toList());
return result;
}
@RequestMapping("/config")
public List<ServiceDTO> getConfigService(
@RequestParam(value = "appId", defaultValue = "") String appId,
@RequestParam(value = "ip", required = false) String clientIp) {
List<InstanceInfo> instances = discoveryService.getConfigServiceInstances();
List<ServiceDTO> result = instances.stream().map(new Function<InstanceInfo, ServiceDTO>() {
@Override
public ServiceDTO apply(InstanceInfo instance) {
ServiceDTO service = new ServiceDTO();
service.setAppName(instance.getAppName());
service.setInstanceId(instance.getInstanceId());
service.setHomepageUrl(instance.getHomePageUrl());
return service;
}
}).collect(Collectors.toList());
return result;
}
@RequestMapping("/admin")
public List<ServiceDTO> getAdminService() {
List<InstanceInfo> instances = discoveryService.getAdminServiceInstances();
List<ServiceDTO> result = instances.stream().map(new Function<InstanceInfo, ServiceDTO>() {
@Override
public ServiceDTO apply(InstanceInfo instance) {
ServiceDTO service = new ServiceDTO();
service.setAppName(instance.getAppName());
service.setInstanceId(instance.getInstanceId());
service.setHomepageUrl(instance.getHomePageUrl());
return service;
}
}).collect(Collectors.toList());
return result;
}
}
-
提供了三个 API ,
services/meta
、services/config
、services/admin
获得 Meta Service、Config Service、Admin Service 集群地址。😈 实际上,services/meta
暂时是不可用的,获取不到实例,因为 Meta Service 目前内嵌在 Config Service 中。 -
在每个 API 中,调用 DiscoveryService 调用对应的方法,获取服务集群。
-
com.ctrip.framework.apollo.core.dto.ServiceDTO
,服务 DTO 。代码如下:public class ServiceDTO { /** * 应用名 */ private String appName; /** * 实例编号 */ private String instanceId; /** * Home URL */ private String homepageUrl; }
3.3 DiscoveryService
@Service
public class DiscoveryService {
@Autowired
private EurekaClient eurekaClient;
public List<InstanceInfo> getConfigServiceInstances() {
Application application = eurekaClient.getApplication(ServiceNameConsts.APOLLO_CONFIGSERVICE);
if (application == null) {
Tracer.logEvent("Apollo.EurekaDiscovery.NotFound", ServiceNameConsts.APOLLO_CONFIGSERVICE);
}
return application != null ? application.getInstances() : Collections.emptyList();
}
public List<InstanceInfo> getMetaServiceInstances() {
Application application = eurekaClient.getApplication(ServiceNameConsts.APOLLO_METASERVICE);
if (application == null) {
Tracer.logEvent("Apollo.EurekaDiscovery.NotFound", ServiceNameConsts.APOLLO_METASERVICE);
}
return application != null ? application.getInstances() : Collections.emptyList();
}
public List<InstanceInfo> getAdminServiceInstances() {
Application application = eurekaClient.getApplication(ServiceNameConsts.APOLLO_ADMINSERVICE);
if (application == null) {
Tracer.logEvent("Apollo.EurekaDiscovery.NotFound", ServiceNameConsts.APOLLO_ADMINSERVICE);
}
return application != null ? application.getInstances() : Collections.emptyList();
}
}
-
每个方法,调用
EurekaClient#getApplication(appName)
方法,获得服务集群。 -
com.ctrip.framework.apollo.core.ServiceNameConsts
,枚举了所有服务的名字。代码如下:public interface ServiceNameConsts { String APOLLO_METASERVICE = "apollo-metaservice"; String APOLLO_CONFIGSERVICE = "apollo-configservice"; String APOLLO_ADMINSERVICE = "apollo-adminservice"; String APOLLO_PORTAL = "apollo-portal"; }
3.4 集群
考虑到高可用,Meta Service 必须集群。因为 Meta Service 自身扮演了目录服务的角色,所以此时不得不引入 Proxy Server 。从选择上,笔者想到的是:
- Nginx ,目前互联网上最常用的 Proxy Server 。
- Zuul ,可以和 Eureka 打通,实现注册与发现。
因为 Meta Service 目前并未注册到 Zuul 上,所以相比来说,Nginx 会是更合适的选择。当然,😈 Nginx 自身也是要做高可用的,哈哈哈,这块胖友自己 Google 下解决方案。
在高性能之前,一切服务节点必须高可用。任何服务节点的松懈,势必在未来的某个时刻,给我们来一波暴击!!!
4. ConfigServiceLocator
在 apollo-client
项目中,com.ctrip.framework.apollo.internals.ConfigServiceLocator
,Config Service 定位器。
- 初始时,从 Meta Service 获取 Config Service 集群地址进行缓存。
- 定时任务,每 5 分钟,从 Meta Service 获取 Config Service 集群地址刷新缓存。
🙂 代码比较简单,胖友自己查看。代码如下:
public class ConfigServiceLocator {
private static final Logger logger = LoggerFactory.getLogger(ConfigServiceLocator.class);
private static final Joiner.MapJoiner MAP_JOINER = Joiner.on("&").withKeyValueSeparator("=");
private static final Escaper queryParamEscaper = UrlEscapers.urlFormParameterEscaper();
private HttpUtil m_httpUtil;
private ConfigUtil m_configUtil;
/**
* ServiceDTO 数组的缓存
*/
private AtomicReference<List<ServiceDTO>> m_configServices;
private Type m_responseType;
/**
* 定时任务 ExecutorService
*/
private ScheduledExecutorService m_executorService;
/**
* Create a config service locator.
*/
public ConfigServiceLocator() {
List<ServiceDTO> initial = Lists.newArrayList();
m_configServices = new AtomicReference<>(initial);
m_responseType = new TypeToken<List<ServiceDTO>>() {}.getType();
m_httpUtil = ApolloInjector.getInstance(HttpUtil.class);
m_configUtil = ApolloInjector.getInstance(ConfigUtil.class);
this.m_executorService = Executors.newScheduledThreadPool(1, ApolloThreadFactory.create("ConfigServiceLocator", true));
// 初始拉取 Config Service 地址
this.tryUpdateConfigServices();
// 创建定时任务,定时拉取 Config Service 地址
this.schedulePeriodicRefresh();
}
/**
* Get the config service info from remote meta server.
*
* @return the services dto
*/
public List<ServiceDTO> getConfigServices() {
// 缓存为空,强制拉取
if (m_configServices.get().isEmpty()) {
updateConfigServices();
}
// 返回 ServiceDTO 数组
return m_configServices.get();
}
private boolean tryUpdateConfigServices() {
try {
updateConfigServices();
return true;
} catch (Throwable ex) {
//ignore
}
return false;
}
private void schedulePeriodicRefresh() {
this.m_executorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
logger.debug("refresh config services");
Tracer.logEvent("Apollo.MetaService", "periodicRefresh");
// 拉取 Config Service 地址
tryUpdateConfigServices();
}
}, m_configUtil.getRefreshInterval(), m_configUtil.getRefreshInterval(), m_configUtil.getRefreshIntervalTimeUnit());
}
private synchronized void updateConfigServices() {
// 拼接请求 Meta Service URL
String url = assembleMetaServiceUrl();
HttpRequest request = new HttpRequest(url);
int maxRetries = 2; // 重试两次
Throwable exception = null;
// 循环请求 Meta Service ,获取 Config Service 地址
for (int i = 0; i < maxRetries; i++) {
Transaction transaction = Tracer.newTransaction("Apollo.MetaService", "getConfigService");
transaction.addData("Url", url);
try {
// 请求
HttpResponse<List<ServiceDTO>> response = m_httpUtil.doGet(request, m_responseType);
transaction.setStatus(Transaction.SUCCESS);
// 获得结果 ServiceDTO 数组
List<ServiceDTO> services = response.getBody();
// 获得结果为空,重新请求
if (services == null || services.isEmpty()) {
logConfigService("Empty response!");
continue;
}
// 更新缓存
m_configServices.set(services);
// 打印结果 ServiceDTO 数组
logConfigServices(services);
return;
} catch (Throwable ex) {
Tracer.logEvent("ApolloConfigException", ExceptionUtil.getDetailMessage(ex));
transaction.setStatus(ex);
exception = ex;
} finally {
transaction.complete();
}
// 请求失败,sleep 等待下次重试
try {
m_configUtil.getOnErrorRetryIntervalTimeUnit().sleep(m_configUtil.getOnErrorRetryInterval());
} catch (InterruptedException ex) {
// ignore
}
}
// 请求全部失败,抛出 ApolloConfigException 异常
throw new ApolloConfigException(String.format("Get config services failed from %s", url), exception);
}
private String assembleMetaServiceUrl() {
String domainName = m_configUtil.getMetaServerDomainName();
String appId = m_configUtil.getAppId();
String localIp = m_configUtil.getLocalIp();
// 参数集合
Map<String, String> queryParams = Maps.newHashMap();
queryParams.put("appId", queryParamEscaper.escape(appId));
if (!Strings.isNullOrEmpty(localIp)) {
queryParams.put("ip", queryParamEscaper.escape(localIp));
}
return domainName + "/services/config?" + MAP_JOINER.join(queryParams);
}
private void logConfigServices(List<ServiceDTO> serviceDtos) {
for (ServiceDTO serviceDto : serviceDtos) {
logConfigService(serviceDto.getHomepageUrl());
}
}
private void logConfigService(String serviceUrl) {
Tracer.logEvent("Apollo.Config.Services", serviceUrl);
}
}
5. AdminServiceAddressLocator
在 apollo-portal
项目中,com.ctrip.framework.apollo.portal.component.AdminServiceAddressLocator
,Admin Service 定位器。
- 初始时,创建延迟 1 秒的任务,从 Meta Service 获取 Config Service 集群地址进行缓存。
- 获取成功时,创建延迟 5 分钟的任务,从 Meta Service 获取 Config Service 集群地址刷新缓存。
- 获取失败时,创建延迟 10 秒的任务,从 Meta Service 获取 Config Service 集群地址刷新缓存。
🙂 代码比较简单,胖友自己查看。代码如下:
@Component
public class AdminServiceAddressLocator {
private static final Logger logger = LoggerFactory.getLogger(AdminServiceAddressLocator.class);
private static final long NORMAL_REFRESH_INTERVAL = 5 * 60 * 1000;
private static final long OFFLINE_REFRESH_INTERVAL = 10 * 1000;
private static final int RETRY_TIMES = 3;
private static final String ADMIN_SERVICE_URL_PATH = "/services/admin";
/**
* 定时任务 ExecutorService
*/
private ScheduledExecutorService refreshServiceAddressService;
private RestTemplate restTemplate;
/**
* Env 数组
*/
private List<Env> allEnvs;
/**
* List<ServiceDTO 缓存 Map
*
* KEY:ENV
*/
private Map<Env, List<ServiceDTO>> cache = new ConcurrentHashMap<>();
@Autowired
private HttpMessageConverters httpMessageConverters; // 暂未使用
@Autowired
private PortalSettings portalSettings;
@Autowired
private RestTemplateFactory restTemplateFactory;
@PostConstruct
public void init() {
// 获得 Env 数组
allEnvs = portalSettings.getAllEnvs();
// init restTemplate
restTemplate = restTemplateFactory.getObject();
// 创建 ScheduledExecutorService
refreshServiceAddressService = Executors.newScheduledThreadPool(1, ApolloThreadFactory.create("ServiceLocator", true));
// 创建延迟任务,1 秒后拉取 Admin Service 地址
refreshServiceAddressService.schedule(new RefreshAdminServerAddressTask(), 1, TimeUnit.MILLISECONDS);
}
public List<ServiceDTO> getServiceList(Env env) {
// 从缓存中获得 ServiceDTO 数组
List<ServiceDTO> services = cache.get(env);
// 若不存在,直接返回空数组。这点和 ConfigServiceLocator 不同。
if (CollectionUtils.isEmpty(services)) {
return Collections.emptyList();
}
// 打乱 ServiceDTO 数组,返回。实现 Client 级的负载均衡
List<ServiceDTO> randomConfigServices = Lists.newArrayList(services);
Collections.shuffle(randomConfigServices);
return randomConfigServices;
}
// maintain admin server address
private class RefreshAdminServerAddressTask implements Runnable {
@Override
public void run() {
boolean refreshSuccess = true;
// 循环多个 Env ,请求对应的 Meta Service ,获得 Admin Service 集群地址
// refresh fail if get any env address fail
for (Env env : allEnvs) {
boolean currentEnvRefreshResult = refreshServerAddressCache(env);
refreshSuccess = refreshSuccess && currentEnvRefreshResult;
}
// 若刷新成功,则创建定时任务,5 分钟后执行
if (refreshSuccess) {
refreshServiceAddressService.schedule(new RefreshAdminServerAddressTask(), NORMAL_REFRESH_INTERVAL, TimeUnit.MILLISECONDS);
// 若刷新失败,则创建定时任务,10 秒后执行
} else {
refreshServiceAddressService.schedule(new RefreshAdminServerAddressTask(), OFFLINE_REFRESH_INTERVAL, TimeUnit.MILLISECONDS);
}
}
}
private boolean refreshServerAddressCache(Env env) {
for (int i = 0; i < RETRY_TIMES; i++) {
try {
// 请求 Meta Service ,获得 Admin Service 集群地址
ServiceDTO[] services = getAdminServerAddress(env);
// 获得结果为空,continue ,继续执行下一次请求
if (services == null || services.length == 0) {
continue;
}
// 更新缓存
cache.put(env, Arrays.asList(services));
// 返回获取成功
return true;
} catch (Throwable e) {
logger.error(String.format("Get admin server address from meta server failed. env: %s, meta server address:%s", env, MetaDomainConsts.getDomain(env)), e);
Tracer.logError(String.format("Get admin server address from meta server failed. env: %s, meta server address:%s", env, MetaDomainConsts.getDomain(env)), e);
}
}
// 返回获取失败
return false;
}
private ServiceDTO[] getAdminServerAddress(Env env) {
String domainName = MetaDomainConsts.getDomain(env); // MetaDomainConsts
String url = domainName + ADMIN_SERVICE_URL_PATH;
return restTemplate.getForObject(url, ServiceDTO[].class);
}
}
5.1 MetaDomainConsts
com.ctrip.framework.apollo.core.MetaDomainConsts
,Meta Service 多环境的地址枚举类。代码如下:
/**
* The meta domain will load the meta server from System environment first, if not exist, will load
* from apollo-env.properties. If neither exists, will load the default meta url.
* <p>
* Currently, apollo supports local/dev/fat/uat/lpt/pro environments.
*/
public class MetaDomainConsts {
private static Map<Env, Object> domains = new HashMap<>();
public static final String DEFAULT_META_URL = "http://config.local";
static {
// 读取配置文件到 Properties 中
Properties prop = new Properties();
prop = ResourceUtils.readConfigFile("apollo-env.properties", prop);
// 获得系统 Properties
Properties env = System.getProperties();
// 添加到 domains 中
// 优先级,env > prop
domains.put(Env.LOCAL, env.getProperty("local_meta", prop.getProperty("local.meta", DEFAULT_META_URL)));
domains.put(Env.DEV, env.getProperty("dev_meta", prop.getProperty("dev.meta", DEFAULT_META_URL)));
domains.put(Env.FAT, env.getProperty("fat_meta", prop.getProperty("fat.meta", DEFAULT_META_URL)));
domains.put(Env.UAT, env.getProperty("uat_meta", prop.getProperty("uat.meta", DEFAULT_META_URL)));
domains.put(Env.LPT, env.getProperty("lpt_meta", prop.getProperty("lpt.meta", DEFAULT_META_URL)));
domains.put(Env.PRO, env.getProperty("pro_meta", prop.getProperty("pro.meta", DEFAULT_META_URL)));
}
public static String getDomain(Env env) {
return String.valueOf(domains.get(env));
}
}
- 具体的读取顺序和说明,见英文注释说明。🙂 英语和我一样有非常大的进步空的同学,可以使用有道词典翻译。
5.2 Env
com.ctrip.framework.apollo.core.enums.Env
,环境枚举。代码如下:
/**
* Here is the brief description for all the predefined environments:
* <ul>
* <li>LOCAL: Local Development environment, assume you are working at the beach with no network access</li>
* <li>DEV: Development environment</li>
* <li>FWS: Feature Web Service Test environment</li>
* <li>FAT: Feature Acceptance Test environment</li>
* <li>UAT: User Acceptance Test environment</li>
* <li>LPT: Load and Performance Test environment</li>
* <li>PRO: Production environment</li>
* <li>TOOLS: Tooling environment, a special area in production environment which allows
* access to test environment, e.g. Apollo Portal should be deployed in tools environment</li>
* </ul>
*
* @author Jason Song(song_s@ctrip.com)
*/
public enum Env {
LOCAL, DEV, FWS, FAT, UAT, LPT, PRO, TOOLS;
public static Env fromString(String env) {
Env environment = EnvUtils.transformEnv(env);
Preconditions.checkArgument(environment != null, String.format("Env %s is invalid", env));
return environment;
}
}
😎 前面一直忘记对 Env 介绍,所以有些奇怪的放在这个位置。主要目的是,我们可以参考携程对服务环境的命名和定义。
666. 彩蛋
😝😝😝 第4、5 小节写的比较简略,如果有不理解的胖友,可以给我的公众号留言。啦啦啦,我去刷《复仇者联盟3》啦,美滋滋。