在一个项目开量验证过程中,发现 createDate 字段不正确,比正确时间晚了十四个小时。调研发现,这是一个非常典型的问题。现在把定位问题的思路和解决办法给大家做个分享。
首先,检查数据库配置,查询线上生产环境配置,结果如下:
图 1. MySQL 变量 同时,检查线上生产环境 MySQL 版本,为问题复现做准备:
图 2. MySQL 版本 从数据库配置上来说,基本正常,没有发现什么问题。(持续运行了这么长时间,有问题应该早就发现了。)
其次,检查数据库连接配置,正式环境的链接配置如下:
jdbc:mysql://<host>:3306/<schema>?createDatabaseIfNotExist=true &characterEncoding=utf-8&useUnicode=true&connectTimeout=2000 &socketTimeout=2000&autoReconnect=true 数据库连接也没有问题。
第三,询问 SA 线上服务器时区配置,回复上是 CST,这个和数据库对应,没有问题。
图 3. 与 SA 沟通 配置检查正常,那么只好在本地搭建环境,重现问题,再寻求解决方案。由于项目是基于 Spring Boot 2.3.7.RELEASE 开发的,相关依赖也尽量使用 Spring Boot 指定版本的,所以,很快把开发环境搭好了。
在配置服务器环境时,遇到一点小小的问题:我一直以为有个时区名称叫 CST,就在网上去查怎么设置,结果徒劳半天也没有找到。后来上开发机检查开发机时区配置,发现是 Asia/Shanghai。将测试服务器设置为该时区,数据库内部查询时区,显示和服务器一直。
调试代码中,发现 MySQL 连接驱动的代码中,有配置时区的相关代码,如下:
com.mysql.cj.protocol.a.NativeProtocol#configureTimezone /** * Configures the client's timezone if required. * * @throws CJException * if the timezone the server is configured to use can't be * mapped to a Java timezone. */ public void configureTimezone() { // 获取服务器时区 String configuredTimeZoneOnServer = this.serverSession.getServerVariable("time_zone"); // 如果服务器时区是 SYSTEM,则使用服务器的 system_time_zone 时区设置 if ("SYSTEM".equalsIgnoreCase(configuredTimeZoneOnServer)) { configuredTimeZoneOnServer = this.serverSession.getServerVariable("system_time_zone"); } // 获取客户端时区配置 String canonicalTimezone = getPropertySet().getStringProperty(PropertyKey.serverTimezone).getValue(); // 如果服务器时区不为空,切客户端时区配置不可用,则使用服务器的时区配置 if (configuredTimeZoneOnServer != null) { // user can override this with driver properties, so don't detect if that's the case if (canonicalTimezone == null || StringUtils.isEmptyOrWhitespaceOnly(canonicalTimezone)) { try { canonicalTimezone = TimeUtil.getCanonicalTimezone(configuredTimeZoneOnServer, getExceptionInterceptor()); } catch (IllegalArgumentException iae) { throw ExceptionFactory.createException(WrongArgumentException.class, iae.getMessage(), getExceptionInterceptor()); } } } if (canonicalTimezone != null && canonicalTimezone.length() > 0) { // 为该会话设置时区 this.serverSession.setServerTimeZone(TimeZone.getTimeZone(canonicalTimezone)); // // The Calendar class has the behavior of mapping unknown timezones to 'GMT' instead of throwing an exception, so we must check for this... // if (!canonicalTimezone.equalsIgnoreCase("GMT") && this.serverSession.getServerTimeZone().getID().equals("GMT")) { throw ExceptionFactory.createException(WrongArgumentException.class, Messages.getString("Connection.9", new Object[] { canonicalTimezone }), getExceptionInterceptor()); } } }
在前面的文章 HikariCP 源码分析 — ConcurrentBag 中,D瓜哥分析了一下 HikariCP 中一个非常重要的数据结构 ConcurrentBag。
今天,继续再介绍 HikariCP 中另一个很关键的数据结构: FastList。
FastList 本身的实现非常简单,要理解它的奥秘,就需要结合 Java 原生集合类的 ArrayList 来比较性地看。
构造函数 先来对比一下两者的构造函数。先来看看 FastList:
FastList public final class FastList<T> implements List<T>, RandomAccess, Serializable { private static final long serialVersionUID = -4598088075242913858L; private final Class<?> clazz; private T[] elementData; private int size; /** * Construct a FastList with a default size of 32. * @param clazz the Class stored in the collection */ @SuppressWarnings("unchecked") public FastList(Class<?> clazz) { this.elementData = (T[]) Array.newInstance(clazz, 32); this.clazz = clazz; } /** * Construct a FastList with a specified size. * @param clazz the Class stored in the collection * @param capacity the initial size of the FastList */ @SuppressWarnings("unchecked") public FastList(Class<?> clazz, int capacity) { this.elementData = (T[]) Array.newInstance(clazz, capacity); this.clazz = clazz; }
在上一篇文章 分布式锁之 Apache Curator InterProcessMutex 中介绍了基于 ZooKeeper 实现的互斥锁。除此之外,还可以实现读写锁。这篇文章就来简要介绍一下 InterProcessReadWriteLock 的实现原理。
老规矩,先看看类的注释:
/** * <p> * A re-entrant read/write mutex that works across JVMs. Uses Zookeeper to hold the lock. All processes * in all JVMs that use the same lock path will achieve an inter-process critical section. Further, this mutex is * "fair" - each user will get the mutex in the order requested (from ZK's point of view). * </p> * * <p> * A read write lock maintains a pair of associated locks, one for read-only operations and one * for writing. The read lock may be held simultaneously by multiple reader processes, so long as * there are no writers. The write lock is exclusive. * </p> * * <p> * <b>Reentrancy</b><br> * This lock allows both readers and writers to reacquire read or write locks in the style of a * re-entrant lock. Non-re-entrant readers are not allowed until all write locks held by the * writing thread/process have been released. Additionally, a writer can acquire the read lock, but not * vice-versa. If a reader tries to acquire the write lock it will never succeed.<br><br> * * <b>Lock downgrading</b><br> * Re-entrancy also allows downgrading from the write lock to a read lock, by acquiring the write * lock, then the read lock and then releasing the write lock. However, upgrading from a read * lock to the write lock is not possible. * </p> */ public class InterProcessReadWriteLock {
对分布式锁耳熟能详。不过,一直关注的是基于 Redis 实现的分布式锁。知道 ZooKeeper 也可以实现分布式锁。但是,原来的想法是把 Redis 那个思路切换到 ZooKeeper 上来实现就好。今天了解到 Apache Curator 内置了分布式锁的实现: InterProcessMutex。查看了一下源码实现,发现跟基于 Redis 实现的源码相比,在思路上还是有很大不同的。所以,特别作文记录一下。
先来看一下,整体流程:
结合流程图和源码,加锁的过程是这样的:
先判断本地是否有锁数据,如果有则对锁定次数自增一下,然后返回 true;
如果没有锁数据,则尝试获取锁:
在指定路径下创建临时顺序节点
获取指定路径下,所有节点,检查自身是否是序号最小的节点:
如果自身序号最小,则获得锁;否则
如果自身不是序号最小的节点,则通过 while 自旋 + wait(times) 不断尝试获取锁,直到成功。
获得锁后,把锁信息缓存在本地 ConcurrentMap<Thread, LockData> threadData 变量中,方便计算重入。
在 ZooKeeper 中的结构大致如下:
下面我们逐个方法进行分析说明。先来看一下 InterProcessMutex 的注释:
/** * A re-entrant mutex that works across JVMs. Uses Zookeeper to hold the lock. All processes in all JVMs that * use the same lock path will achieve an inter-process critical section. Further, this mutex is * "fair" - each user will get the mutex in the order requested (from ZK's point of view) */ public class InterProcessMutex implements InterProcessLock, Revocable<InterProcessMutex>
以前无意间搜资料了解到 HikariCP,一下子就被它的简洁代码和卓越性能吸引住了。以前也有翻过它的代码,但是不是很系统,最近再次翻阅,正好做些笔记,方便以后学习。
D瓜哥最近在学习 Java 并发知识。那就从 HikariCP 自定义的并发集合 ConcurrentBag 开始学习。
在 HikariCP 的 Wiki 中,有 Down the Rabbit Hole · ConcurrentBag 的章节来专门介绍 ConcurrentBag:
ConcurrentBag 的灵感借鉴自 C# .NET 的 ConcurrentBag 类。但是实现却是完全不同的。这里的 ConcurrentBag 有如下特性:
A lock-free design
ThreadLocal caching
Queue-stealing
Direct hand-off optimizations
下面,通过代码来对此做个说明。
在 ConcurrentBag 类的定义中,声明了集合元素必须是 IConcurrentBagEntry 的子类。先来看看这个接口的定义:
public interface IConcurrentBagEntry { int STATE_NOT_IN_USE = 0; int STATE_IN_USE = 1; int STATE_REMOVED = -1; int STATE_RESERVED = -2; boolean compareAndSet(int expectState, int newState); void setState(int newState); int getState(); } 接下来,看一下成员变量:
// 存放共享元素 private final CopyOnWriteArrayList<T> sharedList; private final boolean weakThreadLocals; // 在 ThreadLocal 缓存线程本地元素,避免线程争用 private final ThreadLocal<List<Object>> threadList; private final IBagStateListener listener; // private final AtomicInteger waiters; private volatile boolean closed; // 接力队列 private final SynchronousQueue<T> handoffQueue; 在 ConcurrentBag 开头的 JavaDoc 中就做了明确说明:
Note that items that are "borrowed" from the bag are not actually removed from any collection, so garbage collection will not occur even if the reference is abandoned. Thus care must be taken to "requite" borrowed objects otherwise a memory leak will result. Only the "remove" method can completely remove an object from the bag.