Clojurians Log v2

As someone who comes from the land of single threadedness, I have been playing around with multi-threaded examples in my spare time to grow my understanding and uncover my assumptions about multi-threaded environments. One such experiment is a cache for CompletableFutures. If I just use ConcurrentHashMap#computeIfAbsent, my machine can run one billion iterations of a simple completable future in about 22 seconds. However, the moment I add eviction logic, that time increases to about 36 minutes. Is there any way I can make the following more efficient? As far as I can tell, synchronization is required for this logic, so I’m not sure that it can be improved (at least not by much):

public CompletableFuture<T> removeIfExpired(String key) {
	synchronized(cache) {
		CacheEntry<T> cached = cache.get(key);
		if (cached == null || cached.getExpiresInMs() == null) {
			return null;
		}

		if (Duration.between(cached.getCreatedAt(), Instant.now()).toMillis() >= cached.getExpiresInMs()) {
			CacheEntry<T> removed = cache.remove(key);
			if (removed != cached) {
				throw new IllegalStateException("Concurrency error: item removed from cache not the original item checked for expiration.");
			}
			return removed.getValue();
		}

		return null;
	}
}