一、引言 Huazie 的 flea-framework 框架下的 flea-cache ,我们已经介绍了有关 Memcached 接入 和 Redis 接入 ;那么本篇我们将要介绍如何 整合接入 Memcached 和 Redis 。
二、往期文章
三、主要内容 1. 参考 flea-cache使用之整合Memcached和Redis接入 源代码
2. 依赖 Memcached-Java-Client-3.0.2.jar
1 2 3 4 5 6 <dependency > <groupId > com.whalin</groupId > <artifactId > Memcached-Java-Client</artifactId > <version > 3.0.2</version > </dependency >
jedis-3.0.1.jar
1 2 3 4 5 6 <dependency > <groupId > redis.clients</groupId > <artifactId > jedis</artifactId > <version > 3.0.1</version > </dependency >
spring-context-4.3.18.RELEASE.jar
1 2 3 4 5 6 <dependency > <groupId > org.springframework</groupId > <artifactId > spring-context</artifactId > <version > 4.3.18.RELEASE</version > </dependency >
spring-context-support-4.3.18.RELEASE.jar
1 2 3 4 5 <dependency > <groupId > org.springframework</groupId > <artifactId > spring-context-support</artifactId > <version > 4.3.18.RELEASE</version > </dependency >
3. 基础接入 经过上两篇博文的介绍,Memcached 和 Redis 相信很多笔友都能成功的接入应用系统了。随着业务的复杂度上升,缓存的应用场景不断增多,单独的对接一个缓存系统,已经无法满足业务发展要求。
本文着眼于整合多套缓存接入:一个缓存 cache 对应一个缓存数据 cache-data ,一个缓存数据 cache-data 对应一个缓存组 cache-group ,多个缓存服务器 cache-server 关联一个缓存组 cache-group ,一个缓存组 cache-group 对应具体的缓存接入实现(目前支持 Memcached 和 Redis )。
下面且听我慢慢道来:
3.1 Flea缓存配置文件 Flea缓存配置文件 ( flea-cache-config.xml ),用来整合 Memcached 和 Redis 的相关配置,包含了缓存数据,缓存组,缓存服务器,缓存参数以及其他缓存配置项。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 <?xml version="1.0" encoding="UTF-8" ?> <flea-cache-config > <cache-items key ="FleaCacheInit" desc ="缓存初始化配置项" > <cache-item key ="systemName" desc ="缓存所属系统名" > FleaFrame</cache-item > </cache-items > <cache-items key ="FleaCacheBuilder" desc ="Flea缓存建造者实现" > <cache-item key ="MemCached" desc ="MemCached的Flea缓存建造者实现" > com.huazie.fleaframework.cache.memcached.builder.MemCachedFleaCacheBuilder</cache-item > <cache-item key ="RedisSharded" desc ="Redis分片模式下的Flea缓存建造者实现" > com.huazie.fleaframework.cache.redis.builder.RedisShardedFleaCacheBuilder</cache-item > <cache-item key ="RedisCluster" desc ="Redis集群模式下的Flea缓存建造者实现" > com.huazie.fleaframework.cache.redis.builder.RedisClusterFleaCacheBuilder</cache-item > </cache-items > <cache-params > <cache-param key ="fleacore.nullCacheExpiry" desc ="空缓存数据有效期(单位:s)" > 10</cache-param > <cache-param key ="redis.connectionTimeout" desc ="Redis客户端socket连接超时时间(单位:ms)" > 2000</cache-param > <cache-param key ="redis.soTimeout" desc ="Redis客户端socket读写超时时间(单位:ms)" > 2000</cache-param > <cache-param key ="redis.hashingAlg" desc ="Redis分布式hash算法(1:MURMUR_HASH,2:MD5)" > 1</cache-param > <cache-param key ="redis.pool.maxTotal" desc ="Redis客户端Jedis连接池最大连接数" > 100</cache-param > <cache-param key ="redis.pool.maxIdle" desc ="Redis客户端Jedis连接池最大空闲连接数" > 10</cache-param > <cache-param key ="redis.pool.minIdle" desc ="Redis客户端Jedis连接池最小空闲连接数" > 0</cache-param > <cache-param key ="redis.pool.maxWaitMillis" desc ="Redis客户端Jedis连接池获取连接时的最大等待时间(单位:ms)" > 2000</cache-param > <cache-param key ="redis.maxAttempts" desc ="Redis客户端操作最大尝试次数【包含第一次操作】" > 5</cache-param > <cache-param key ="redis.cluster.connectionTimeout" desc ="Redis客户端socket连接超时时间(单位:ms)" > 2000</cache-param > <cache-param key ="redis.cluster.soTimeout" desc ="Redis客户端socket读写超时时间(单位:ms)" > 2000</cache-param > <cache-param key ="redis.cluster.password" desc ="Redis集群服务节点登录密码(集群各节点配置同一个)" > huazie123</cache-param > <cache-param key ="memcached.initConn" desc ="初始化时对每个服务器建立的连接数目" > 20</cache-param > <cache-param key ="memcached.minConn" desc ="每个服务器建立最小的连接数" > 20</cache-param > <cache-param key ="memcached.maxConn" desc ="每个服务器建立最大的连接数" > 500</cache-param > <cache-param key ="memcached.maintSleep" desc ="自查线程周期进行工作,其每次休眠时间(单位:ms)" > 60000</cache-param > <cache-param key ="memcached.nagle" desc ="Socket的参数,如果是true在写数据时不缓冲,立即发送出去" > true</cache-param > <cache-param key ="memcached.socketTO" desc ="Socket阻塞读取数据的超时时间(单位:ms)" > 3000</cache-param > <cache-param key ="memcached.socketConnectTO" desc ="Socket连接超时时间(单位:ms)" > 3000</cache-param > <cache-param key ="memcached.hashingAlg" desc ="MemCached分布式hash算法" > 3</cache-param > </cache-params > <cache-datas > <cache-data type ="fleaAuth" desc ="Flea Auth缓存数据所在组配置" > authGroup</cache-data > <cache-data type ="fleaFrame" desc ="Flea Frame配置数据所在组配置" > configGroup</cache-data > <cache-data type ="fleaJersey" desc ="Flea Jersey配置数据所在组配置" > configGroup</cache-data > <cache-data type ="fleaDynamic" desc ="Flea 动态数据缓存所在组配置" > dynamicGroup</cache-data > </cache-datas > <cache-groups > <cache-group group ="authGroup" desc ="Flea权限数据缓存组" > MemCached</cache-group > <cache-group group ="configGroup" desc ="Flea配置数据缓存组" > RedisSharded</cache-group > <cache-group group ="dynamicGroup" desc ="Flea动态数据缓存组" > RedisCluster</cache-group > </cache-groups > <cache-servers > <cache-server group ="authGroup" weight ="1" desc ="MemCached缓存服务器" > 127.0.0.1:31113</cache-server > <cache-server group ="authGroup" weight ="1" desc ="MemCached缓存服务器" > 127.0.0.1:31114</cache-server > <cache-server group ="configGroup" password ="huazie123" weight ="1" desc ="Redis缓存服务器【分片模式】" > 127.0.0.1:10001</cache-server > <cache-server group ="configGroup" password ="huazie123" weight ="1" desc ="Redis缓存服务器【分片模式】" > 127.0.0.1:10002</cache-server > <cache-server group ="configGroup" password ="huazie123" weight ="1" desc ="Redis缓存服务器【分片模式】" > 127.0.0.1:10003</cache-server > <cache-server group ="dynamicGroup" desc ="Redis缓存服务器【集群模式】" > 127.0.0.1:20011</cache-server > <cache-server group ="dynamicGroup" desc ="Redis缓存服务器【集群模式】" > 127.0.0.1:20012</cache-server > <cache-server group ="dynamicGroup" desc ="Redis缓存服务器【集群模式】" > 127.0.0.1:20021</cache-server > <cache-server group ="dynamicGroup" desc ="Redis缓存服务器【集群模式】" > 127.0.0.1:20022</cache-server > <cache-server group ="dynamicGroup" desc ="Redis缓存服务器【集群模式】" > 127.0.0.1:20031</cache-server > <cache-server group ="dynamicGroup" desc ="Redis缓存服务器【集群模式】" > 127.0.0.1:20032</cache-server > </cache-servers > </flea-cache-config >
3.2 Flea缓存定义文件 Flea缓存定义文件(flea-cache.xml ),用来定义各类缓存,其中 key 表示缓存主关键字, type 表示一类缓存数据,expiry 表示缓存生效时长(单位:秒【0:永久】)。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 <?xml version="1.0" encoding="UTF-8" ?> <flea-cache > <caches > <cache key ="fleaconfigdata" type ="fleaFrame" expiry ="86400" desc ="Flea配置数据缓存" /> <cache key ="fleajerseyi18nerrormapping" type ="fleaJersey" expiry ="86400" desc ="Flea Jersey 国际码和错误码映射缓存" /> <cache key ="fleajerseyresservice" type ="fleaJersey" expiry ="86400" desc ="Flea Jersey 资源服务缓存" /> <cache key ="fleajerseyresclient" type ="fleaJersey" expiry ="86400" desc ="Flea Jersey 资源客户端缓存" /> <cache key ="fleajerseyresource" type ="fleaJersey" expiry ="86400" desc ="Flea Jersey 资源缓存" /> <cache key ="fleamenufavorites" type ="fleaDynamic" expiry ="0" desc ="Flea菜单收藏夹数据缓存" /> </caches > <import resource ="flea/cache/flea-auth-cache.xml" /> </flea-cache >
3.3 定义核心Flea缓存类 CoreFleaCache 同样继承抽象Flea缓存 AbstractFleaCache ,实现其定义的抽象方法;内部定义成员变量 fleaCache 用于指定具体的 Flea 缓存实现(这个具体的实现,可参考 Memcached接入 和 Redis分片模式接入 ),实现的三个方法 getNativeValue ,putNativeValue ,deleteNativeValue 内部采用具体Flea缓存实现fleaCache相应的方法实现读缓存、写缓存,删缓存;从构造方法可见,fleaCache 通过 FleaCacheFactory.getFleaCache(name) ,从Flea缓存工厂中获取。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 public class CoreFleaCache extends AbstractFleaCache { private static final FleaLogger LOGGER = FleaLoggerProxy.getProxyInstance(CoreFleaCache.class); private AbstractFleaCache fleaCache; public CoreFleaCache (String name) { super (name, CacheConfigUtils.getExpiry(name), CacheConfigUtils.getNullCacheExpiry()); fleaCache = FleaCacheFactory.getFleaCache(name); cache = fleaCache.getCache(); } @Override public Object getNativeValue (String key) { if (LOGGER.isDebugEnabled()) { LOGGER.debug1(new Object () {}, "KEY = {}" , key); } return fleaCache.getNativeValue(key); } @Override public Object putNativeValue (String key, Object value, int expiry) { if (LOGGER.isDebugEnabled()) { Object obj = new Object () {}; LOGGER.debug1(obj, "CORE FLEA CACHE, KEY = {}" , key); LOGGER.debug1(obj, "CORE FLEA CACHE, VALUE = {}" , value); LOGGER.debug1(obj, "CORE FLEA CACHE, EXPIRY = {}s" , expiry); LOGGER.debug1(obj, "CORE FLEA CACHE, NULL CACHE EXPIRY = {}s" , getNullCacheExpiry()); } return fleaCache.putNativeValue(key, value, expiry); } @Override public Object deleteNativeValue (String key) { if (LOGGER.isDebugEnabled()) { LOGGER.debug1(new Object () {}, "KEY = {}" , key); } return fleaCache.deleteNativeValue(key); } @Override public String getSystemName () { return CacheConfigUtils.getSystemName(); } }
3.4 定义Flea缓存工厂类 FleaCacheFactory 根据缓存名(即缓存主关键字)所创建的Flea缓存都存入 ConcurrentMap<String, AbstractFleaCache> 中。newCache 方法主要是根据缓存名查找相关缓存 cache ,再找到缓存数据 cache-data ,接着找到缓存组 cache-group ,最后根据缓存组所属的缓存系统,查找缓存配置项 cache-item ,获取对应Flea缓存的建造者实现,可参见 flea-cache-config.xml 配置。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 public class FleaCacheFactory { private static final ConcurrentMap<String, AbstractFleaCache> fleaCacheMap = new ConcurrentHashMap <>(); private FleaCacheFactory () { } public static AbstractFleaCache getFleaCache (String name) { if (!fleaCacheMap.containsKey(name)) { synchronized (fleaCacheMap) { if (!fleaCacheMap.containsKey(name)) { fleaCacheMap.put(name, newFleaCache(name)); } } } return fleaCacheMap.get(name); } private static AbstractFleaCache newFleaCache (String name) { } }
3.5 定义Flea缓存建造者接口类 IFleaCacheBuilder 定义 build
方法,用于构建 Flea
缓存对象
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 public interface IFleaCacheBuilder { AbstractFleaCache build (String name, List<CacheServer> cacheServerList) ; }
3.6 定义Memcached Flea缓存建造者 MemCachedFleaCacheBuilder 实现 IFleaCacheBuilder ,用于构建基于 Memcached 的 Flea 缓存,即创建一个 MemCachedFleaCache 。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 public class MemCachedFleaCacheBuilder implements IFleaCacheBuilder { private static final FleaLogger LOGGER = FleaLoggerProxy.getProxyInstance(MemCachedFleaCacheBuilder.class); @Override public AbstractFleaCache build (String name, List<CacheServer> cacheServerList) { if (CollectionUtils.isEmpty(cacheServerList)) { throw new FleaCacheConfigException ("无法初始化MemCached Flea缓存,缓存服务器列表【cacheServerList】为空" ); } int expiry = CacheConfigUtils.getExpiry(name); int nullCacheExpiry = CacheConfigUtils.getNullCacheExpiry(); String group = cacheServerList.get(0 ).getGroup(); MemCachedClient memCachedClient = new MemCachedClient (group); MemCachedPool.getInstance(group).initialize(cacheServerList); AbstractFleaCache fleaCache = new MemCachedFleaCache (name, expiry, nullCacheExpiry, memCachedClient); if (LOGGER.isDebugEnabled()) { Object obj = new Object () {}; LOGGER.debug1(obj, "Pool Name = {}" , MemCachedPool.getInstance(group).getPoolName()); LOGGER.debug1(obj, "Pool = {}" , MemCachedPool.getInstance(group).getSockIOPool()); } return fleaCache; } }
3.7 定义Redis Flea缓存建造者 RedisShardedFleaCacheBuilder 实现 IFleaCacheBuilder ,用于构建基于Redis的Flea缓存,即创建一个 分片模式的 RedisFleaCache 。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 public class RedisShardedFleaCacheBuilder implements IFleaCacheBuilder { private static final FleaLogger LOGGER = FleaLoggerProxy.getProxyInstance(RedisShardedFleaCacheBuilder.class); @Override public AbstractFleaCache build (String name, List<CacheServer> cacheServerList) { if (CollectionUtils.isEmpty(cacheServerList)) { throw new FleaCacheConfigException ("无法初始化分片模式下Redis Flea缓存,缓存服务器列表【cacheServerList】为空" ); } int expiry = CacheConfigUtils.getExpiry(name); int nullCacheExpiry = CacheConfigUtils.getNullCacheExpiry(); String group = cacheServerList.get(0 ).getGroup(); RedisShardedPool.getInstance(group).initialize(cacheServerList); RedisClient redisClient = RedisClientFactory.getInstance(group); AbstractFleaCache fleaCache = new RedisFleaCache (name, expiry, nullCacheExpiry, CacheModeEnum.SHARDED, redisClient); if (LOGGER.isDebugEnabled()) { Object obj = new Object () {}; LOGGER.debug1(obj, "Pool Name = {}" , RedisShardedPool.getInstance(group).getPoolName()); LOGGER.debug1(obj, "Pool = {}" , RedisShardedPool.getInstance(group).getJedisPool()); } return fleaCache; } }
3.8 定义Redis集群模式Flea缓存建造者 RedisClusterFleaCacheBuilder 实现 IFleaCacheBuilder ,用于构建基于Redis的Flea缓存,即创建一个 集群模式的 RedisFleaCache 。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 public class RedisClusterFleaCacheBuilder implements IFleaCacheBuilder { private static final FleaLogger LOGGER = FleaLoggerProxy.getProxyInstance(RedisClusterFleaCacheBuilder.class); @Override public AbstractFleaCache build (String name, List<CacheServer> cacheServerList) { if (CollectionUtils.isEmpty(cacheServerList)) { throw new FleaCacheConfigException ("无法初始化集群模式下Redis Flea缓存,缓存服务器列表【cacheServerList】为空" ); } int expiry = CacheConfigUtils.getExpiry(name); int nullCacheExpiry = CacheConfigUtils.getNullCacheExpiry(); String group = cacheServerList.get(0 ).getGroup(); RedisClusterPool.getInstance(group).initialize(cacheServerList); RedisClient redisClient = RedisClientFactory.getInstance(group, CacheModeEnum.CLUSTER); AbstractFleaCache fleaCache = new RedisFleaCache (name, expiry, nullCacheExpiry, CacheModeEnum.CLUSTER, redisClient); if (LOGGER.isDebugEnabled()) { Object obj = new Object () {}; LOGGER.debug1(obj, "Pool Name = {}" , RedisClusterPool.getInstance(group).getPoolName()); LOGGER.debug1(obj, "JedisCluster = {}" , RedisClusterPool.getInstance(group).getJedisCluster()); } return fleaCache; } }
3.9 定义核心Flea缓存管理类 CoreFleaCacheManager 继承 AbstractFleaCacheManager ,实现 newCache 方法,用于创建一个核心 Flea 缓存。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 public class CoreFleaCacheManager extends AbstractFleaCacheManager { @Override protected AbstractFleaCache newCache (String name, int expiry) { return new CoreFleaCache (name); } }
3.10 整合接入自测 单元测试类 FleaCacheTest
首先,这里需要按照 Flea缓存配置文件 (flea-cache-config.xml ) 中的缓存服务器 cache-server 中地址部署相应的 Memcached 和 Redis 服务,可参考笔者的 这篇博文 。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 @Test public void testCoreFleaCache () { try { AbstractFleaCacheManager manager = FleaCacheManagerFactory.getFleaCacheManager(CacheEnum.FleaCore.getName()); AbstractFleaCache cache = manager.getCache("fleaconfigdata" ); LOGGER.debug("Cache={}" , cache); cache.get("menu1" ); cache.get("menu2" ); cache.getCacheKey(); LOGGER.debug(cache.getCacheName() + ">>>" + cache.getCacheDesc()); } catch (Exception e) { LOGGER.error("Exception:" , e); } }
经过上面的介绍,核心Flea缓存相关的内容,基本上算是讲解完毕。在不改变现有业务代码的基础上,相关缓存 cache 可以通过修改其归属的缓存数据类型 type ,实现各类缓存数据,多种缓存系统之间的无缝迁移。
4. 进阶接入 4.1 定义核心Spring缓存类 CoreSpringCache 继承抽象 Spring 缓存 AbstractSpringCache ,用于对接 Spring ;从构造方法可见,该类初始化使用核心Flea缓存类 CoreFleaCache 。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 public class CoreSpringCache extends AbstractSpringCache { public CoreSpringCache (String name, IFleaCache fleaCache) { super (name, fleaCache); } public CoreSpringCache (String name) { super (name, new CoreFleaCache (name)); } }
4.2 定义核心Spring缓存管理类 CoreSpringCacheManager 继承抽象 Spring 缓存管理类 AbstractSpringCacheManager ,用于对接 Spring ;基本实现同核心 Flea 缓存管理类 CoreFleaCacheManager ,唯一不同在于 newCache 的实现,这边是 new 一个核心 Spring 缓存 CoreSpringCache 。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 public class CoreSpringCacheManager extends AbstractSpringCacheManager { @Override protected AbstractSpringCache newCache (String name, int expiry) { return new CoreSpringCache (name); } }
4.3 Spring配置 1 2 3 4 5 <bean id ="coreSpringCacheManager" class ="com.huazie.fleaframework.cache.core.manager.CoreSpringCacheManager" /> <cache:annotation-driven cache-manager ="coreSpringCacheManager" proxy-target-class ="true" />
4.4 缓存自测 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 private ApplicationContext applicationContext; @Before public void init () { applicationContext = new ClassPathXmlApplicationContext ("applicationContext.xml" ); LOGGER.debug("ApplicationContext={}" , applicationContext); } @Test public void testCoreSpringCache () { try { AbstractSpringCacheManager manager = (CoreSpringCacheManager) applicationContext.getBean("coreSpringCacheManager" ); LOGGER.debug("CoreSpringCacheManager={}" , manager); AbstractSpringCache cache = manager.getCache("fleaparadetail" ); LOGGER.debug("Cache={}" , cache); cache.clear(); } catch (Exception e) { LOGGER.error("Exception:" , e); } }
4.5 业务逻辑层接入缓存管理 @Cacheable 使用,value 为缓存名,也作缓存主关键字, key 为具体的缓存键
1 2 3 4 5 6 7 8 9 10 11 12 @Cacheable(value = "fleaparadetail", key = "#paraType + '_' + #paraCode") public FleaParaDetail getParaDetail (String paraType, String paraCode) throws Exception { List<FleaParaDetail> fleaParaDetails = fleaParaDetailDao.getParaDetail(paraType, paraCode); FleaParaDetail fleaParaDetailValue = null ; if (CollectionUtils.isNotEmpty(fleaParaDetails)) { fleaParaDetailValue = fleaParaDetails.get(0 ); } return fleaParaDetailValue; }
四、结语 到目前为止,整合 Memcached 和 Redis 接入的工作已经全部完成,相信各位已经能够接入系统~~~