Release Notes - ZooKeeper - Version 3.7.0
New Feature
- - Add support for C client for SASL authentication
- - The benchmark tools for zookeeper
- - Enforce the quota limit
- - Add s390x support for Travis build
- - Add (Cyrus) SASL authentication support to Perl client
- - Official API to start ZooKeeper server from Java
- - Introduce a deterministic runtime behavior injection framework for ZooKeeperServer testing
- - Allow multiple superUsers with SASL
- - Add whoami API and Cli command
- - Optionally canonicalize host names in quorum SASL authentication
Improvement
- - Add an option to zkCli to wait for connection before executing commands
- - Code clean up in ZooKeeperServer and KerberosName
- - The ZooKeeper do not write in log session ID in which the client has been authenticated.
- - Add option to not set ACL for reconfig node
- - Add comment reminding users to add cases to zerror when adding values to ZOO_ERRORS
- - Use StandardCharsets
- - remove the deprecated CLI: ls2 and rmr
- - Introduce SnapshotComparer that assists debugging with snapshots.
- - SASL (Kerberos) Authentication with SSL for clients and Quorum
- - Generalize target authentication scheme for ZooKeeper authentication enforcement.
- - Add SSL support for the zk python client
- - use factory design pattern to refactor ZooKeeperMain
- - refactor the async api call to lambda style
- - Update Jetty to 9.4.24.v20191120
- - Implement "batch mode" in cli_mt
- - ls -s CLI need a line break
- - Remove NPE Possibility in Follower Class
- - Clean Up ZNodeName Class
- - remove the deprecated LogFormatter tool
- - Use ThreadLocalConcurrent to Replace Random and Math.random
- - Remove Redundant GroupID from Maven POMs
- - Upgrade maven-compiler-plugin For ZooKeeper-jute
- - Stop initializing new SSL connection if ZK server is shutting down
- - Discard requests that are delayed longer than a configured threshold
- - Jute Use JDK hashCode Methods for Native Types
- - Use StandardCharsets UTF-8 in Jute toString
- - Improving leader efficiency via not processing learner's requests in commit processor
- - Use JDK String Join Method in ZK StringUtils
- - Use Map computeIfAbsent in AvgMinMaxCounterSet Class
- - Move Logging Code into Logging Guard in Learner
- - make logs of ResponseCache more readable
- - move traceMask calculation logic into the trace log in the FinalRequestProcessor#processRequest
- - Fix ZooKeeper 3.5 C client build on Fedora8
- - upgrade JLine jar dependency
- - fix a large amount of maven build warnings
- - Make sources buildable with JDK14
- - Simplify generation of VersionInfoMain and Info
- - Add m2e configuration in pom.xml for Eclipse developers
- - zkpython: Minor compilation and testing issues
- - Miscellaneous Maven improvements
- - Skip Learner Request made to ObserverMaster from going to next processor
- - NIOServerCnxnFactory static block has no used code
- - correct the documentation about digest.enabled
- - cleaning up the code,A static field should be directly referred by its class name
- - Add a test that does a minimal validation of Apache Curator
- - Do Not Override Plugin Versions from Apache Parent POM
- - Use Commons and JDK Functions in ClientBase
- - ReconfigBackupTest Remove getFileContent
- - new UncaughtExceptionHandler object with lambda
- - Enhance documentation for property ssl.clientAuth
- - Upgrade to Netty 4.1.50.Final
- - Add ARM64 jobs to Travis-CI
- - make the rc constant in the ClientCnxn
- - upgrade dependency-check to version 6.0.0
- - Handle float metrics in check_zookeeper
- - Upgrade commons-cli to 1.4
- - Add support for BCFKS key/trust store format
- - Remove commons-lang from ZooKeeper
- - Remove json-simple from ZooKeeper
- - Update dependency versions and eliminate java docs warnings
- - Update ZooKeeper client documentation about key file format parameters
- - Auto close resources with try catch block
- - Adding additional security metrics to zookeeper
- - GenerateLoad needs to use log for protecting sensitive data
- - use the computeIfAbsent to simplify the Leader#processSync method
- - Remove unnecessary judgment of null
- - Upgrade Mockito to 3.6.28 - allow builds on JDK16
- - Update checkstyle-strict.xml by the latest version 8.39 of checkstyle
- - add a doc about whoami CLI
- - Update Netty version to 4.1.53.Final on 3.5 branch
- - Improve the error message when message goes above jute.maxbufer size
- - Add document for snapshot compression config
Bug
- - c client zookeeper_close not send CLOSE_OP request to server
- - Misuse of INET_ADDRSTRLEN
- - C library calls getaddrinfo unconditionally from zookeeper_interest
- - fast leader election keeps failing
- - ZooKeeper not starting because acceptedEpoch is less than the currentEpoch
- - Include ZKClientConfig API in zoookeeper javadoc
- - infinitely connect on windows
- - QuorumCnxManager.Listener Thread Better handling of SocketTimeoutException
- - fd leak due to UnresolvedAddressException on connect.
- - Handle Java 9/11 additions of covariant return types to java.nio.ByteBuffer methods
- - ZK prime_connection(the Handshake) can complete without reading all the payload.
- - handle NPE gracefully when the watch parameter of zookeeper java client is null
- - ZKConfig fails to return proper value on getBoolean() when user accidentally includes spaces at the end of the value
- - Data inconsistency when the leader crashes right after sending SNAP sync
- - Data loss after upgrading standalone ZK server 3.4.14 to 3.5.6 with snapshot.trust.empty=true
- - NettyServerCnxnFactoryTest is flaky
- - Audit Log feature fails in a stand alone zookeeper setup
- - Incorrect *_CFLAGS handling in Automake
- - SyncRequestProcessor doesn't update lastFlushTime correctly on observers
- - set jute.maxbuffer hexadecimal number throw parseInt error
- - NoRouteToHostException when starting large ZooKeeper cluster on localhost
- - upgrade jackson-databind to address CVE-2019-20330
- - Split brain on log disk full
- - [trivial bug] fix compile error in PurgeTxnTest introduced by ZOOKEEPER-3231
- - invalid ipv6 address comparison in C client
- - Unable to eliminate log4j1 transitive dependency
- - Avoid use of broken codehaus properties-maven-plugin
- - Remove use of com.sun.nio.file.SensitivityWatchEventModifier
- - Update copyright notices from 2019 to 2020
- - Resolve release requirements in download page
- - fast leader election does not end if leader is taken down
- - JettyAdminServer should not allow HTTP TRACE method
- - restore Version.getRevision() to be backward compatible
- - Zookeeper 3.5.7 not creating snapshot
- - Replace filter with list comprehension for returning list in zk-merge-pr.py
- - Request throttling is broken when RequestThrottler is disabled or configured incorrectly.
- - Fix Jenkins link in pom
- - ZooKeeper config propagates even with disabled dynamic reconfig
- - fix zkServer.sh status command to support SSL-only server
- - Zookeeper refuses request after node expansion
- - After add a new node, zookeeper cluster won't commit any proposal if this new node is leader
- - ZKHostnameVerifier rejects valid certificates with subjectAltNames
- - Rolling scale up of zookeeper cluster does not work with reconfigEnabled=false
- - Do not track global sessions in ReadOnlyZooKeeperServer
- - fix backward-compatibility for ZooKeeperServer constructor
- - zkServer.sh status command fails when IPV6 is configured
- - JMX Bean RemotePeerBean should enclose IPV6 host in square bracket same as LocalPeerBean
- - Client connection fails if IPV6 is not enclosed in square brackets
- - zoo_aremove_watches segfault: zk_hashtable needs locking!
- - ZKCli commands give wrong error message "Authentication is not valid" for insufficient permissions
- - Client side NullPointerException in case of empty Multi operation
- - Race condition causes sessions to be created for clients even though their certificate authentication has failed
- - Data inconsistency caused by DIFF sync uncommitted log
- - owasp failing with json-simple-1.1.1.jar: CVE-2020-10663, CVE-2020-7712
- - C client: avoid out-of-order packets during SASL negotiation
- - Zookeeper Inspector throwing NullPointerExceptions and not displaying properly
- - zookeeper c api sasl client memory leak
- - Compile Error in Zookeeper.c without SASL
- - use of uninitialized data in zookeeper-client/zookeeper-client-c/src/zookeeper.c:free_auth_completion
- - added a shebang or a 'shell' directive to lastRevision.sh
- - Clients can corrupt the audit log
- - C client test suite hangs forever 'sss' is configured in /etc/nsswitch.conf
- - Build failures when running surefire tests concurrently due to bind address already in use
- - QuorumCnxManager Listener port bind retry does not retry DNS lookup
- - addWatch api should check the null watch
- - disconnect reason wrong
- - CVE-2020-25649 - Upgrade jackson databind to 2.10.5.1
- - Zookeeper Inspector reports "List of default node viewers is empty" when not specifically run from the zookeeper-contrib/zookeeper-contrib-zooinspector directory
- - Dockerfile can't build Zookeeper C client library
- - Missing executable bits in source release tarball
- - Avoid thread leak in QuorumRequestPipelineTest
- - WatcherCleanerTest often fails on macOS Catalina
- - C client: SASL-related compilation issues on macOS Catalina
- - Test fails when port 8080 is in use
- - New CI pipeline checks out master in branch builds too
- - Quota checks break setData in multi transactions
- - Potential redundant connection attempts during leader election
- - Use dynamic temp folder instead of static temp folder in RestMain
- - InvalidSnapshotTest corrupts its own test data
Test
- - test
Wish
- - convert internal logic to use java 8 streams
- - Restore ZKUtil.deleteRecursive in order to help compatibility of applications with 3.5 and 3.6
Task
- - Use switch Statement in ClientCnxn SendThread
- - owasp checker failing for - CVE-2019-17571 Apache Log4j 1.2 deserialization of untrusted data in SocketServer
- - Source release tarball does not match repository in 3.6.0
- - Support alternative algorithms for ACL digest
- - upgrade maven dependency-check to 5.3.0
- - Fix issues reported in 3.6.0rc3
- - upgrade jackson-databind to address CVE-2020-8840
- - upgrade jackson-databind to 2.10 from 2.9
- - upgrade netty to address CVE-2020-11612
- - owasp failing due to CVE-2020-9488
- - Migrate Jenkins jobs to ci-hadoop.apache.org
- - Create Owasp check build on new Jenkins instance
- - Create .asf.yaml file for ZooKeeper repo
- - Jetty License Update
- - Create configuration files GitHub Actions CI builds
- - Fix Jenkinsfiles with new tool names
- - Flaky test MultipleAddressTest::testGetValidAddressWithNotValid
- - Owasp check failing - Jetty 9.4.32 - CVE-2020-27216
- - dependency-check:check failing - Jetty 9.4.34.v20201102 - CVE-2020-27218
- - Update copyright notices from 2020 to 2021
- - dependency-check:check failing - Jetty 9.4.35.v20201120 - CVE-2020-27223
Sub-task
- - cyclic dependency ClientCnxn, ZooKeeper
- - Close quorum socket asynchronously to avoid server shutdown stalled by long socket closing time
- - Moving sending packets in Learner to a separate thread
- - Add metric JVM_PAUSE_TIME
- - Upgrade jUnit in ZooKeeper-Jute
- - Upgrade jUnit in ZooKeeper-Recipes
- - Upgrade jUnit in ZooKeeper-Metrics-providers
- - Add a couple metrics to track inflight diff syncs and snap syncs
- - Add a couple request processor metrics
- - Re-enable deprecation check after finishing jUnit upgrade
- - Upgrade jUnit in ZooKeeper-server
- - Update hamcrest-library to version 2.2