New Avatica Repository

Roughly one year after Avatica’s Maven build was separated from Calcite’s (as described in this previous post), Avatica now has its own Git repository.

The Apache Calcite PMC felt like it was a good time to lift the Avatica project out of the Calcite repository, given the continue maturation of Avatica. For developers and contributors to Avatica, nothing will functionally change, except for the location of the code itself.

The de-facto repository can be found at the ASF’s Git hosting, with a mirrored-copy also available on Github at apache/calcite-avatica.

Release 1.9.0

Apache Calcite Avatica 1.9.0 includes various improvements to make it more robust and secure, while maintaining API and protocol compatibility with previous versions.

We now include non-shaded and shaded artifacts, to make it easier to embed Avatica in your application.

There are improvements to the JDBC API, adding support for canceling statements, and improving type conversions and metadata.

The transport is upgraded to use protobuf-3.1.0 (previously 3.0 beta); we have also upgraded Jetty.

See the release notes; download the release.

Release 1.8.0

Apache Calcite Avatica 1.8.0 continues the focus on compatibility with previous versions while also adding support for authentication between Avatica client and server. Performance, notably on the write-path, is also major area of improvement in this release, increasing as much as two to three times over previous versions with the addition of new API support. The documentation for both users and developers continues to receive improvements.

Authentication is a major theme of this release, providing multiple layers of additional authentication mechanisms over previous versions. In these earlier versions, the only authentication provided by Avatica was achieved via the JDBC URL’s standard user and password options. These have always been passed directly into the backend database’s authentication system, but not all databases provide username and password based authentication systems. CALCITE-1173 adds Avatica-level authentication over HTTP Basic and HTTP Digest authentication mechanisms. These are provided specifically for the case when Avatica is used with a database that does not already provide its own authentication implementation.

Some systems rely on Kerberos for strong, centrally- managed authentication. CALCITE-1159 introduces Kerberos-based authentication for clients via SPNEGO. The Avatica server can be configured to only allow clients with a valid Kerberos ticket, optionally, also passing this information to the backend database to implement basic “impersonation” (where the Avatica server issues requests on behalf of the end user).

Building on top of the work done in Avatica-1.7.0 in CALCITE-1091, this release also contains CALCITE-1128 which implements the batch-oriented JDBC APIs on Statement. Through careful inspection, it was observed that the overall performance of Avatica clients in 100% write workloads was dominated by the cost of the HTTP calls. By leveraging the Statement#addBatch() and Statement#executeBatch() API calls, clients can efficiently batch multiple updates in a single HTTP call. In testing this over the previous single HTTP call per update with Apache Phoenix, it was observed that performance increased by two to three times, bringing Avatica’s performance on par with the Phoenix “native” driver.

Returning back to compatibility, a new component appears in this release which is designed to test versions of Avatica against each other. CALCITE-1190 introduces a “Technology Compatibility Kit” (TCK) which automates the testing of one version of Avatica against other versions. To further ease this testing, a runnable JAR to launch an HSQLDB instance and an Avatica server also makes it debut with these changes. This TCK makes it much easier to run tests of newer clients against older servers and vice versa. Validating the backwards compatibility that is being built is extremely important to be confident in the guarantees being provided to users.

Finally, a number of bugs are also fixed in the Protocol Buffer wire API. Some of these include CALCITE-1113 and CALCITE-1103 which fix how certain numbers are serialized, CALITE-1243 which corrects some fields in Protocol Buffer messages which were incorrectly marked as unsigned integers instead of signed integers, and CALCITE-1209 which removes incorrect parsing of binary fields as Base64-encoded strings. All of these issues are fixed in a backwards-compatible manner and should have no additional negative impact on older clients (older clients will not break, but they may continue to return incorrect data for certain numbers).

For users of the Avatica driver, a new client reference page is added which details the options that are available in the Avatica JDBC Driver’s URL. The wire API documentation for Protocol Buffers continues to receive updates as the API continues to evolve.

See the release notes; download the release.

Release 1.7.1

Apache Calcite’s PMC has just released Avatica 1.7.1, the first release of Avatica as an independent project. We’re excited because it makes Avatica easier to use, and allows us to be more agile in our release schedule.

(Avatica was previously released as part of Calcite. Avatica is still governed by Apache Calcite’s PMC, and stored in the same git repository as Calcite, but releases are no longer synchronized, and Avatica does not depend on any Calcite modules.)

A significant portion of the work of this release was “Maven work” to separate the build and release processes, but there were several important bug fixes, including a security fix for Jetty (see below).

Performance, specifically on the write path, was a big focus in this release. [CALCITE-1091] contained a number of important changes. Some of these changes (e.g. [CALCITE-1092] and [CALCITE-1093]) were related to heap usage in the Avatica server, while [CALCITE-1094] and [CALCITE-1117] were strictly performance-related.

The latter improved performance the most. Switching to the Apache Commons HttpComponents Client library instead of using the Java platform’s built-in HttpURLConnection, we noticed a 15% improvement in pure write workloads.

Three dependent library upgrades:

  • We completely removed Commons Logging in favor of SLF4J in [CALCITE-669]. This logging framework update will allow downstream integrators to use the logging implementation of their choice instead of being forced to inherit Commons Logging.
  • We upgraded Jackson from 2.1.1 to 2.6.3 in [CALCITE-1021].
  • We upgraded Jetty from 9.2.7.v20150116 to 9.2.15.v20160210 in [CALCITE-1156] to fix a security issue.

Note that Avatica’s Maven coordinates have changed. The groupId is now “org.apache.calcite.avatica” (previously “org.apache.calcite”), and artifactIds are “avatica”, “avatica-metrics”, “avatica-metrics-dropwizardmetrics3”, “avatica-noop-driver”, “avatica-server” (previously “calcite-avatica”, etc.). Make sure to update these when upgrading to this version.

See the release notes; download the release.

Splitting Avatica from Calcite

This marks the separation of Avatica from its previous location as a sub-module of Apache Calcite’s Maven build.

This separation is not to remove Avatica from the governance of the Apache Calcite project, but to allow for even more rapid releases from both the Avatica and Calcite projects. We can confidently make new releases of each without having to worry about the current state of development features in the other.

Calcite Twitter

The official @ApacheCalcite Twitter account pushes announcements about Calcite. If you give a talk about Calcite, let us know and we'll tweet it out and add it to the news section of the website.