History

For a full list of releases, see github. Downloads are available on the downloads page.

1.8.0 / 2016-06-13

This release adds adapters for Elasticsearch and Druid. It is also now easier to make a JDBC connection based upon a single adapter.

There are several new SQL features: UNNEST with multiple arguments, MAP arguments and with a JOIN; a DESCRIBE statement; and a TRANSLATE function like the one in Oracle and PostgreSQL. We also added support for SELECT without FROM (equivalent to the VALUES clause, and widely used in MySQL and PostgreSQL), and added a conformance parameter to allow you to selectively enable this and other SQL features.

And a couple of dozen bug-fixes and enhancements to planner rules and APIs.

Compatibility: This release is tested on Linux, Mac OS X, Microsoft Windows; using Oracle JDK 1.7, 1.8; Guava versions 14.0 to 19.0; other software versions as specified in pom.xml.

New features

  • [CALCITE-1177] Extend list of supported time units in EXTRACT, CEIL and FLOOR functions (Venki Korukanti)
  • [CALCITE-1259] Allow connecting to a single schema without writing a model
  • [CALCITE-750] Support aggregates within windowed aggregates (Gautam Parai)
  • [CALCITE-1250] UNNEST applied to MAP data type (Johannes Schulte)
  • [CALCITE-1253] Elasticsearch adapter (Subhobrata Dey)
  • [CALCITE-1228] Bind parameters in INSERT
  • [CALCITE-1120] SELECT without FROM (Jimmy Xiang)
  • [CALCITE-855] UNNEST with multiple arguments
  • [CALCITE-1225] UNNEST with JOIN
  • [CALCITE-1115] Add TRANSLATE function with 3 parameters, like the one in Oracle (Javanshir Yelchiyev)
  • [CALCITE-1168] Add DESCRIBE statement (Arina Ielchiieva)
  • [CALCITE-1121] Druid adapter
    • [CALCITE-1276] In Druid adapter, deduce tables and columns if not specified
  • [CALCITE-1207] Allow numeric connection properties, and ‘K’, ‘M’, ‘G’ suffixes

Planner rules

  • [CALCITE-1235] Fully push down LIMIT + OFFSET in Cassandra
  • [CALCITE-1216] Rule to convert Filter-on-Scan to materialized view (Amogh Margoor)
  • [CALCITE-1200] Extend RelOptUtil.splitJoinCondition to handle IS NOT DISTINCT FROM (Venki Korukanti)
  • [CALCITE-1211] Allow free use of CassandraSort for LIMIT
  • [CALCITE-1210] Allow UUID filtering in Cassandra
  • [CALCITE-1182] Add ProjectRemoveRule to pre-processing program of materialization substitution

Bug fixes, API changes and minor enhancements

  • [CALCITE-1281] Druid adapter wrongly returns all numeric values as int or float
  • [CALCITE-1279] Druid “select” query gives ClassCastException
  • [CALCITE-1277] Rat fails on source distribution due to git.properties
  • Update KEYS
  • [CALCITE-1252] Handle ANY type in RexBuilder.ensureType and TypeFactory.leastRestrictive (Mehand Baid, Minji Kim)
  • [CALCITE-1151] Fix SqlSetOption to correctly handle SqlOperator.createCall (Sudheesh Katkam, Minji Kim)
  • [CALCITE-1106] Expose constructor for ProjectJoinTransposeRule (Minji Kim)
  • [CALCITE-1107] Make SqlSumEmptyIsZeroAggFunction constructor public (Minji Kim)
  • [CALCITE-1269] Replace IntList with Guava Ints class
  • [CALCITE-1239] Upgrade to avatica-1.8.0
  • [CALCITE-1266] RelBuilder.field gets offsets wrong
  • [CALCITE-1264] Litmus argument interpolation (Chris Baynes)
  • [CALCITE-1245] Allow RelBuilder.scan to take qualified table name (Chris Baynes)
  • Move code from Enumerables to EnumerableDefaults
  • [CALCITE-1246] Cleanup duplicate variables in JoinPushThroughJoinRule (Yi Xinglu)
  • [CALCITE-1241] Add a Freemarker variable for adding non reserved keyword list to Parser.jj template (Venki Korukanti)
    • Create a table in SqlParserTest of reserved keywords from various versions of the SQL standard
  • [CALCITE-1150] Add dynamic record type and dynamic star for schema-on-read table
  • [CALCITE-1238] Unparsing a query with LIMIT but no ORDER BY gives invalid SQL (Emmanuel Bastien)
  • [CALCITE-1230] Add SQLSTATE reference data as enum SqlState in Avatica, and deprecate SqlStateCodes in Calcite
  • [CALCITE-1199] Incorrect trimming of CHAR when performing cast to VARCHAR
  • [CALCITE-1219] Add method SqlOperatorBinding.isOperandLiteral() (Hsuan-Yi Chu)
  • [CALCITE-1222] DatabaseMetaData.getColumnLabel returns null when query has ORDER BY
  • [CALCITE-1215] Fix missing override in CassandraTable
  • [CALCITE-1212] Fix NPE on some Cassandra projects
  • Remove trailing spaces from all source files
  • Move HTTP utilities from Splunk adapter to core
  • Test case for [PHOENIX-2767], non-constant in IN
  • [CALCITE-1166] Disallow sub-classes of RelOptRuleOperand
  • Remove all calls to deprecated methods
  • Add class ConsList
  • More of [CALCITE-999] Clean up maven POM files
  • [CALCITE-1204] Fix invalid Javadoc and suppress checkstyle “errors”
  • [CALCITE-1170] Allow SqlSetOperator to be overridden, as a regular SqlOperator can (Hsuan-Yi Chu)
  • [CALCITE-746] Allow apache-rat to be run outside of release process

Web site and documentation

  • [CALCITE-1273] Following [CALCITE-306], update references to EnumerableTableAccessRel to EnumerableTableScan
  • Fix typo in SQL (Yi Xinglu)
  • Site: add committer, and post slides/video from Polyalgebra talk
  • [CALCITE-1203] Update to github-pages-67
  • [CALCITE-1202] Lock version of Jekyll for bundler
  • Site: add upcoming talks, and fix link to Apache phonebook

1.7.0 / 2016-03-22

This is the first Apache Calcite release since Avatica became an independent project. Calcite now depends on Avatica in the same way as it does other libraries, via a Maven dependency. To see Avatica-related changes, see the release notes for Avatica 1.7.1.

We have added an adapter for Apache Cassandra. You can map a Cassandra keyspace into Calcite as a schema, Cassandra CQL tables as tables, and execute SQL queries on them, which Calcite converts into CQL. Cassandra can define and maintain materialized views but the adapter goes further: it can transparently rewrite a query to use a materialized view even if the view is not mentioned in the query.

This release adds an Oracle-compatibility mode. If you add fun=oracle to your JDBC connect string, you get all of the standard operators and functions plus Oracle-specific functions DECODE, NVL, LTRIM, RTRIM, GREATEST and LEAST. We look forward to adding more functions, and compatibility modes for other databases, in future releases.

We’ve replaced our use of JUL (java.util.logging) with SLF4J. SLF4J provides an API which Calcite can use independent of the logging implementation. This ultimately provides additional flexibility to users, allowing them to configure Calcite’s logging within their own chosen logging framework. This work was done in [CALCITE-669].

For users experienced with configuring JUL in Calcite previously, there are some differences as some the JUL logging levels do not exist in SLF4J: FINE, FINER, and FINEST, specifically. To deal with this, FINE was mapped to SLF4J’s DEBUG level, while FINER and FINEST were mapped to SLF4J’s TRACE.

Compatibility: This release is tested on Linux, Mac OS X, Microsoft Windows; using Oracle JDK 1.7, 1.8; Guava versions 12.0.1 to 19.0; other software versions as specified in pom.xml.

New features

  • [CALCITE-1124] Add TIMESTAMPADD, TIMESTAMPDIFF functions (Arina Ielchiieva)
  • [CALCITE-1066] Add Oracle function table, and functions DECODE, NVL, LTRIM, RTRIM, GREATEST, LEAST
  • [CALCITE-1080] Cassandra adapter (Michael Mior)
  • [CALCITE-1062] In validation, lookup a (possibly overloaded) operator from an operator table (Hsuan-Yi Chu)
  • [CALCITE-551] Sub-query inside aggregate function

Planner rules

  • [CALCITE-1158] Make AggregateRemoveRule extensible
  • [CALCITE-1116] Extend simplify for reducing expressions
  • [CALCITE-1104] Materialized views in Cassandra (Michael Mior)
  • [CALCITE-1130] Add support for operators IS NULL and IS NOT NULL in RexImplicationChecker (Amogh Margoor)
  • [CALCITE-1129] Extend JoinUnionTransposeRule to match Union instead of LogicalUnion (Vasia Kalavri)
  • [CALCITE-1109] Fix up condition when pushing Filter through Aggregate (Amogh Margoor)
  • [CALCITE-1100] If constant reduction no-ops, don’t create a new RelNode (Hsuan-Yi Chu)
  • [CALCITE-1076] Update RelMdDistribution to match other metadata APIs (Ted Xu)
  • [CALCITE-1056] In RelBuilder, simplify predicates, and optimize away WHERE FALSE
  • [CALCITE-1059] Not valid to convert Aggregate on empty to empty if its GROUP BY key is empty

Bug fixes, API changes and minor enhancements

  • [CALCITE-1147] Allow multiple providers for the same kind of metadata
  • [CALCITE-1153] Invalid cast created during SQL Join in Oracle (Chris Atkinson)
  • [CALCITE-1146] Wrong path in CSV example model (wanglan)
  • [CALCITE-1156] Increase Jetty version to 9.2.15.v20160210
  • [CALCITE-1064] Address problematic maven-remote-resources-plugin
  • In TimeUnit add WEEK, QUARTER, MICROSECOND values, and change type of multiplier
  • Deprecate SqlLiteral.SqlSymbol; SqlSymbol can now wrap any enum
  • [CALCITE-1078] Detach avatica from the core calcite Maven project
    • [CALCITE-1077] Switch Calcite to the released Avatica 1.7.1
    • Update groupId when Calcite POMs reference Avatica modules
    • [CALCITE-1137] Exclude Avatica from Calcite source release
  • [CALCITE-1111] Upgrade Guava, and test on a range of Guava versions
  • [CALCITE-1054] Wrong code generation for TIMESTAMP values that may be NULL
  • [CALCITE-604] Tune metadata by generating a dispatcher at runtime
  • [CALCITE-1063] Flat lists for 4, 5, 6 elements
  • Add Orinoco schema (streaming retail data), accessible from Quidem scripts
  • [CALCITE-1097] Exception when executing query with too many aggregation columns (chenzhifa)
  • Add tests for leap days
  • [CALCITE-553] In maven, enable compiler profiles by default
  • [CALCITE-1031] In prepared statement, CsvScannableTable.scan is called twice
  • [CALCITE-1046] Matchers for testing SQL query results
  • [CALCITE-1083] SqlNode.equalsDeep has O(n ^ 2) performance
  • [CALCITE-998] Exception when calling STDDEV_SAMP, STDDEV_POP (Matthew Shaer)
  • [CALCITE-1071] Improve hash functions
  • [CALCITE-1072] CSV adapter incorrectly parses TIMESTAMP values after noon (Chris Albright)
  • [CALCITE-669] Mass removal of Java Logging for SLF4J
  • [CALCITE-1068] Deprecate Stacks
  • [CALCITE-1067] Test failures due to clashing temporary table names
  • [CALCITE-864] Correlation variable has incorrect row type if it is populated by right side of a Join
  • [CALCITE-1021] Upgrade Jackson
  • [CALCITE-999] Clean up maven POM files

Web site and documentation

  • [CALCITE-1112] “Powered by Calcite” page
  • Add SQL-Gremlin to Adapters page
  • [CALCITE-1090] Revise Streaming SQL specification
  • Appoint Josh Elser to PMC
  • Add “Streaming SQL” talk
  • [CALCITE-623] Add a small blurb to the site about Jenkins for CI
  • [CALCITE-1070] Upgrade to new Apache logo
  • Document how to announce a release
  • [CALCITE-1074] Delete old releases from mirroring system

1.6.0 / 2016-01-22

As usual in this release, there are new SQL features, improvements to planning rules and Avatica, and lots of bug fixes. We’ll spotlight a couple of features make it easier to handle complex queries.

[CALCITE-816] allows you to represent sub-queries (EXISTS, IN and scalar) as RexSubQuery, a kind of expression in the relational algebra. Until now, the sql-to-rel converter was burdened with expanding sub-queries, and people creating relational algebra directly (or via RelBuilder) could only create ‘flat’ relational expressions. Now we have planner rules to expand and de-correlate sub-queries.

Metadata is the fuel that powers query planning. It includes traditional query-planning statistics such as cost and row-count estimates, but also information such as which columns form unique keys, unique and what predicates are known to apply to a relational expression’s output rows. From the predicates we can deduce which columns are constant, and following [CALCITE-1023] we can now remove constant columns from GROUP BY keys.

Metadata is often computed recursively, and it is hard to safely and efficiently calculate metadata on a graph of RelNodes that is large, frequently cyclic, and constantly changing. [CALCITE-794] introduces a context to each metadata call. That context can detect cyclic metadata calls and produce a safe answer to the metadata request. It will also allow us to add finer-grained caching and further tune the metadata layer.

Compatibility: This release is tested on Linux, Mac OS X, Microsoft Windows; using Oracle JDK 1.7, 1.8; other software versions as specified in pom.xml.

New features

  • [CALCITE-816] Represent sub-query as a RexNode
  • [CALCITE-854] Implement UNNEST ... WITH ORDINALITY
  • [CALCITE-1003] Utility to convert RelNode to SQL (Amogh Margoor)
    • [CALCITE-1010] FETCH/LIMIT and OFFSET in RelToSqlConverter (Amogh Margoor)
    • Move code from JdbcImplementor and JdbcRules to new class SqlImplementor
    • Deduce dialect’s null collation from DatabaseMetaData
    • Fix RelToSqlConverterTest on Windows
  • Following [CALCITE-897], empty string for boolean properties means true
  • [CALCITE-992] Validate and resolve sequence reference as a Table object
  • [CALCITE-968] Stream-to-relation and stream-to-stream joins (Milinda Pathirage)
  • [CALCITE-1041] User-defined function that returns DATE or TIMESTAMP value
  • [CALCITE-986] User-defined function with DATE or TIMESTAMP parameters
  • [CALCITE-958] Overloaded Table Functions with named arguments (Julien Le Dem)
  • [CALCITE-970] If NULLS FIRST/NULLS LAST not specified, sort NULL values high

Avatica features and bug fixes

  • [CALCITE-1040] Differentiate better between arrays and scalars in protobuf
  • [CALCITE-934] Use an OS-assigned ephemeral port for CalciteRemoteDriverTest
  • [CALCITE-767] Create Avatica RPC endpoints for commit and rollback commands
  • [CALCITE-983] Handle nulls in ErrorResponse’s protobuf representation better
  • [CALCITE-989] Add server’s address in each response
  • Fix some bugs found by static analysis
  • Make all equals and hashCode methods uniform
  • [CALCITE-962] Propagate the cause, not just the cause’s message, from JdbcMeta

Planner rules

  • [CALCITE-1057] Add RelMetadataProvider parameter to standard planner Programs
  • [CALCITE-1055] SubQueryRemoveRule should create Correlate, not Join, for correlated sub-queries
  • [CALCITE-978] Enable customizing constant folding rule behavior when a Filter simplifies to false (Jason Altekruse)
  • [CALCITE-977] Make the constant expression Executor configurable in FrameworkConfig (Jason Altekruse)
  • [CALCITE-1058] Add method RelBuilder.empty, and rewrite LIMIT 0 and WHERE FALSE to it
  • [CALCITE-996] Simplify predicate when we create a Filter operator
  • Simplify RexProgram, in particular (NOT CASE ... END) IS TRUE, which occurs in when NOT IN is expanded
  • Fix variant of [CALCITE-923] that occurs in RelOptRulesTest.testPushFilterPastProject
  • [CALCITE-1023] and [CALCITE-1038] Planner rule that removes Aggregate keys that are constant
  • [CALCITE-1018] SortJoinTransposeRule not firing due to getMaxRowCount(RelSubset) returning null
  • [CALCITE-1019] RelMdUtil.checkInputForCollationAndLimit() was wrong with alreadySorted check
  • Not safe to use ‘=’ for predicates on constant expressions that might be null
  • [CALCITE-993] Pull up all constant expressions, not just literals, as predicates
  • [CALCITE-1005] Handle null in getMaxRowCount for Aggregate (Mike Hinchey)
  • [CALCITE-995] Sort transpose rules might fall in an infinite loop
  • [CALCITE-987] Pushing LIMIT 0 results in an infinite loop (Pengcheng Xiong)
  • [CALCITE-988] FilterToProjectUnifyRule.invert(MutableRel, MutableRel, MutableProject) works incorrectly
  • [CALCITE-969] Composite EnumerableSort with DESC wrongly sorts NULL values low
  • [CALCITE-959] Add description to SortProjectTransposeRule’s constructor

Bug fixes, API changes and minor enhancements

  • [CALCITE-1060] Fix test deadlock by initializing DriverManager before registering AlternatingDriver
  • [CALCITE-1047] ChunkList.clear throws AssertionError
  • [CALCITE-1053] CPU spin, ReflectiveRelMetadataProvider.apply waiting for HashMap.get
  • Upgrade toolbox, to fix line length issue on Windows
  • [CALCITE-1051] Underflow exception due to scaling IN clause literals (Frankie Bollaert)
  • [CALCITE-975] Allow Planner to return validated row type together with SqlNode
  • [CALCITE-1020] Add MILLISECOND in TimeUnit (Pengcheng Xiong)
  • [CALCITE-794] Detect cycles when computing statistics (This is a breaking change.)
  • Tune algorithm that deduces the return type of AND expression
  • [CALCITE-842] Decorrelator gets field offsets confused if fields have been trimmed
  • Fix NullPointerException in SqlJoin.toString()
  • Add ImmutableBitSet.rebuild()
  • [CALCITE-915] Tests now unset ThreadLocal values on exit
  • [CALCITE-1036] DiffRepository should not insert new resources at the end of the repository
  • [CALCITE-955] Litmus (continuation-passing style for methods that check invariants)
  • RelBuilder.project now does nothing if asked to project the identity with the same field names
  • Deprecate some Util methods, and upgrade last Maven modules to JDK 1.7
  • Document RelOptPredicateList
  • Add ImmutableNullableList.copyOf(Iterable)
  • Fix “endPosTable already set” error from javac
  • Add benchmark of Parser.create(sql).parseQuery()
  • [CALCITE-1042] Ensure that FILTER is BOOLEAN NOT NULL
  • [CALCITE-1039] Assign a SqlKind value for each built-in aggregate function
  • [CALCITE-1030] JSON ModelHandler calling SchemaPlus.setCacheEnabled() causes UnsupportedOperationException when using SimpleCalciteSchema
  • [CALCITE-1028] Move populate materializations after sql-to-rel conversion
  • [CALCITE-1034] Use a custom checker for code style rules that Checkstyle cannot express
  • [CALCITE-1032] Verify javadoc of private methods
  • [CALCITE-1015] OFFSET 0 causes AssertionError (Zhen Wang)
  • [CALCITE-1024] In a planner test, if a rule should have no effect, state that explicitly
  • [CALCITE-1016] GROUP BY *constant* on empty relation should return 0 rows
  • [CALCITE-1022] Rename .oq Quidem files to .iq
  • [CALCITE-980] Fix AND and OR implementation in Enumerable convention
  • [CALCITE-459] When parsing SQL, allow single line comment on last line (Zhen Wang)
  • [CALCITE-1009] SelfPopulatingList is not thread-safe
  • [CALCITE-1008] Replace Closeable with AutoCloseable
  • [CALCITE-1001] Upgrade to quidem-0.7
  • [CALCITE-990] In VolcanoPlanner, populate RelOptRuleCall.nodeInputs for operands of type “any”
  • [CALCITE-966] VolcanoPlanner now clears ruleNames in order to avoid rule name conflicting error
  • Factor user-defined function tests from JdbcTest to UdfTest, and classes into Smalls
  • [CALCITE-974] Exception while validating DELETE (Yuri Au Yong)
  • [CALCITE-964] Rename timezone connection property to timeZone

Web site and documentation

  • Avatica
    • [CALCITE-1033] Introduce Avatica protobuf documentation
    • [CALCITE-1029] Add “purpose” descriptions to Avatica JSON docs
    • [CALCITE-984] Massive cleanup of Avatica JSON docs
  • [CALCITE-861] Be explicit that mvn test needs to be invoked
  • [CALCITE-997] Document keywords
  • [CALCITE-979] Broken links in web site
  • [CALCITE-961] Web site: Add downloads and Apache navigation links
  • [CALCITE-960] Download links for pgp, md5, KEYS files, and direct from mirrors
  • Remove embedded date-stamps from javadoc; add javadoc for test classes
  • [CALCITE-965] Link to downloads page from each release news item

1.5.0 / 2015-11-06

Our first release as a top-level Apache project!

Avatica has undergone major improvements, including a new RPC layer that uses protocol buffers, support for DML statements, better support for bind variables and unique identifiers for connections and statements.

There are lots of improvements to planner rules, and the logic that replaces relational expressions with equivalent materializations.

We continue to find more uses for RelBuilder. We now recommend that you use RelBuilder whenever you create relational expressions within a planner rule; the rule can then be re-used to create different sub-classes of relational expression, and the builder will perform simple optimizations automatically.

Using RelBuilder we built Piglet, a subset of the classic Hadoop language Pig. Pig is particularly interesting because it makes heavy use of nested multi-sets. You can follow this example to implement your own query language, and immediately taking advantage of Calcite’s back-ends and optimizer rules. It’s all just algebra, after all!

New features

  • [CALCITE-911] Add a variant of CalciteSchema that does not cache sub-objects
  • [CALCITE-845] Derive SUM’s return type by a customizable policy (Maryann Xue)
  • [CALCITE-916] Support table function that implements ScannableTable
    • Example table function that generates mazes and their solutions
  • [CALCITE-941] Named, optional and DEFAULT arguments to function calls; support named arguments when calling table functions and table macros
  • [CALCITE-910] Improve handling of ARRAY, MULTISET, STRUCT types
  • [CALCITE-879] COLLECT aggregate function
  • [CALCITE-546] Allow table, column and field called ‘*’
  • [CALCITE-893] Theta join in JDBC adapter
  • Linq4j: Implement EnumerableDefaults methods (MiNG)
  • [CALCITE-823] Add ALTER ... RESET statement (Sudheesh Katkam)
  • [CALCITE-881] Allow schema.table.column references in GROUP BY
  • [CALCITE-852] DDL statements
  • [CALCITE-851] Add original SQL string as a field in the parser
  • [CALCITE-819] Add RelRoot, a contract for the result of a relational expression

Avatica features and bug fixes

  • [CALCITE-951] Print the server-side stack in the local exception (Josh Elser)
  • [CALCITE-936] Make HttpServer configurable (Navis Ryu)
  • [CALCITE-903] Enable Avatica client to recover from missing server-side state (Josh Elser)
  • [CALCITE-921] Fix incorrectness when calling getString() on binary data (Josh Elser)
  • [CALCITE-913] Construct proper ColumnMetaData for arrays (Josh Elser)
  • [CALCITE-871] In JdbcMeta, register each statement using an id from a generator (Bruno Dumon)
  • [CALCITE-645] Implement AvaticaSqlException to pass server-side exception information to clients (Josh Elser)
  • [CALCITE-912] Add Avatica OpenConnectionRequest (Bruno Dumon)
  • [CALCITE-919] Avoid setScale on BigDecimal when scale is 0 (Josh Elser)
  • [CALCITE-927] Call finagle for all calls that return ResultSetResponses (Josh Elser)
  • [CALCITE-705] DML in Avatica, and split Execute out from Fetch request (Yeong Wei)
  • [CALCITE-914] Add JsonSubType for ExecuteResponse, and fix JSON docs (Josh Elser)
  • [CALCITE-905] getTables returns empty result in JdbcMeta (Jan Van Besien)
  • [CALCITE-906] Avatica JdbcMeta statement IDs are not unique
  • [CALCITE-866] Break out Avatica documentation and add JSON reference (Josh Elser)
  • [CALCITE-843] AvaticaConnection.getAutoCommit throws NullPointerException
  • [CALCITE-840] Protocol buffer serialization over HTTP for Avatica Server (Josh Elser)

Materializations

  • [CALCITE-952] Organize applicable materializations in reversed topological order (Maryann Xue)
  • [CALCITE-890] Register all combinations of materialization substitutions (Maryann Xue)
  • [CALCITE-891] When substituting materializations, match TableScan without Project (Maryann Xue)
  • [CALCITE-890] Register all combinations of materialization substitutions (Maryann Xue)
  • [CALCITE-925] Match materialized views when predicates contain strings and ranges (Amogh Margoor)
  • [CALCITE-793] Planner requires unnecessary collation when using materialized view (Maryann Xue)
  • [CALCITE-825] Allow user to specify sort order of an ArrayTable

Planner rules

  • [CALCITE-953] Improve RelMdPredicates to deal with RexLiteral (Pengcheng Xiong)
  • [CALCITE-939] Variant of SortUnionTransposeRule for order-preserving Union (Maryann Xue)
  • [CALCITE-931] Wrong collation trait in SortJoinTransposeRule for right joins (Maryann Xue)
  • [CALCITE-938] More accurate rowCount for Aggregate applied to already unique keys (Maryann Xue)
  • [CALCITE-935] Improve how ReduceExpressionsRule handles duplicate constraints (Pengcheng Xiong)
  • [CALCITE-922] Extract value of an INTERVAL literal (Hsuan-Yi Chu)
  • [CALCITE-889] Implement SortUnionTransposeRule (Pengcheng Xiong)
  • [CALCITE-909] Make ReduceExpressionsRule extensible
  • [CALCITE-856] Make more rules extensible
  • [CALCITE-902] Match nullability when reducing expressions in a Project
  • [CALCITE-895] Simplify “(CASEEND) = constant” inside AND or OR (Hsuan-Yi Chu)
  • [CALCITE-828] Use RelBuilder in rules rather than type-specific RelNode factories
  • [CALCITE-892] Implement SortJoinTransposeRule
  • [CALCITE-876] After pushing LogicalProject past LogicalWindow, adjust references to constants properly (Hsuan-Yi Chu)
  • [CALCITE-844] Push Project through Window (Hsuan-Yi Chu)
  • [CALCITE-841] Redundant windows when window function arguments are expressions (Hsuan-Yi Chu)
  • [CALCITE-846] Push Aggregate with Filter through Union(all)

RelBuilder and Piglet

  • [CALCITE-933] RelBuilder.scan() now gives a nice exception if the table does not exist (Andy Grove)
  • Fix Piglet DUMP applied to multisets and structs
  • Multisets and COLLECT in Piglet
  • [CALCITE-785] Add “Piglet”, a subset of Pig Latin on top of Calcite algebra
  • [CALCITE-869] Add VALUES command to Piglet
  • [CALCITE-868] Add API to execute queries expressed as RelNode
  • In RelBuilder, build expressions by table alias

Bug fixes, API changes and minor enhancements

  • [CALCITE-948] Indicator columns not preserved by RelFieldTrimmer
  • Fix Windows issues (line endings and checkstyle suppressions)
  • [CALCITE-937] User-defined function within view
  • [CALCITE-926] Rules fail to match because of missing link to parent equivalence set (Maryann Xue)
  • [CALCITE-908] Bump protobuf to 3.0.0-beta-1, fix deprecations and update docs (Josh Elser)
  • [CALCITE-932] Fix muddled columns when RelFieldTrimmer is applied to Aggregate
  • [CALCITE-930] Now Calcite is a top-level project, remove references to “incubating”
  • [CALCITE-929] Calls to AbstractRelNode may result in NPE
  • [CALCITE-923] Type mismatch when converting LEFT JOIN to INNER
  • [CALCITE-666] Anti-semi-joins against JDBC adapter give wrong results (Yeong Wei)
  • [CALCITE-918] createProject in RelOptUtil should uniquify field names
  • [CALCITE-792] Obsolete RelNode.isKey and isDistinct methods
  • Allow FlatLists of different length to be compared
  • [CALCITE-898] Type of ‘Java * `INTEGER`' should be `BIGINT`
  • [CALCITE-894] Do not generate redundant column alias for the left relation when translating IN subquery (Maryann Xue)
  • [CALCITE-897] Enable debugging using “-Dcalcite.debug”
  • [CALCITE-885] Add Oracle test environment
  • [CALCITE-888] Overlay window loses PARTITION BY list (Hsuan-Yi Chu)
  • [CALCITE-886] System functions in GROUP BY clause
  • [CALCITE-860] Correct LICENSE file for generated web site
  • [CALCITE-882] Allow web site to be deployed not as the root directory of the web server (Josh Elser)
  • Upgrade parent POM to apache-17
  • [CALCITE-687] Synchronize HSQLDB at a coarse level using a Lock (Josh Elser)
  • [CALCITE-870] Remove copyright content from archers.json
  • Replace Stack with ArrayDeque
  • [CALCITE-874] ReflectiveRelMetadataProvider is not thread-safe
  • Add LogicalWindow.create()
  • Add ImmutableBitSet.get(int, int)
  • [CALCITE-865] Unknown table type causes NullPointerException in JdbcSchema
    • Add table types used by Oracle and DB2
  • [CALCITE-862] JdbcSchema gives NullPointerException on non-standard column type (Marc Prud’hommeaux)
  • [CALCITE-847] AVG window function in GROUP BY gives AssertionError (Hsuan-Yi Chu)
  • [CALCITE-827] Calcite incorrectly permutes columns of OVER query (Hsuan-Yi Chu)
  • [CALCITE-809] TableScan does not support large/infinite scans (Jesse Yates)
  • Lazily create exception only when it needs to be thrown (Marc Prud’hommeaux)
  • [CALCITE-812] Make JSON reader and writer use properly quoted key names (Marc Prud’hommeaux)
  • [CALCITE-820] Validate that window functions have OVER clause (Hsuan-Yi Chu)
  • [CALCITE-824] Type inference when converting IN clause to semijoin (Josh Wills)

1.4.0-incubating / 2015-09-02

In addition to a large number of bug fixes and minor enhancements, this release includes improvements to lattices and materialized views, and adds a builder API so that you can easily create relational algebra expressions.

New features

  • [CALCITE-748] Add RelBuilder, builder for expressions in relational algebra
  • [CALCITE-758] Use more than one lattice in the same query (Rajat Venkatesh)
  • [CALCITE-761] Pre-populated materializations (Maryann Xue)
  • [CALCITE-786] Detect if materialized view can be used to rewrite a query in non-trivial cases (Amogh Margoor)
  • [CALCITE-732] Implement multiple distinct-COUNT using GROUPING SETS
  • Add various BitSet and ImmutableBitSet utilities

Website updates

  • [CALCITE-810] Add committers’ organizations to the web site
  • Add news item (XLDB best lighting talk), and some talks
  • Fix javadoc links
  • Add license notice for web site
  • Wrap file header in HTML comments
  • How to release
  • Move disclaimer out of every page’s footer and into home page and downloads page
  • For web site files, add license headers where possible, apache-rat exclusions otherwise
  • Calcite DOAP
  • [CALCITE-355] Web site

Bug fixes, API changes and minor enhancements

  • [CALCITE-741] Ensure that the source release’s DEPENDENCIES file includes all module dependencies
  • [CALCITE-743] Ensure only a single source assembly is executed
  • [CALCITE-850] Remove push down expressions from FilterJoinRule and create a new rule for it
  • [CALCITE-834] StackOverflowError getting predicates from the metadata provider
  • [CALCITE-833] RelOptUtil.splitJoinCondition incorrectly splits a join condition (Hsuan-Yi Chu)
  • [CALCITE-822] Add a unit test case to test collation of LogicalAggregate
  • [CALCITE-822] Revert incorrect LogicalAggregate collation inferring logic made in [CALCITE-783] (Milinda Pathirage)
  • [CALCITE-826] Use ProjectFactory in AggregateJoinTranposeRule and FilterJoinRule
  • [CALCITE-821] Frameworks gives NPE when FrameworkConfig has no default schema
  • [CALCITE-811] Extend JoinProjectTransposeRule with option to support outer joins
  • [CALCITE-805] Add support for using an alternative grammar specification for left and right curly braces. Additionally, add support for including addition token manager declarations
  • [CALCITE-803] Add MYSQL_ANSI Lexing policy
  • [CALCITE-717] Compare BINARY and VARBINARY on unsigned byte values (Low Chin Wei)
  • [CALCITE-814] RexBuilder reverses precision and scale of DECIMAL literal
  • [CALCITE-813] Upgrade updateCount, maxRows from int to long
  • [CALCITE-714] When de-correlating, push join condition into subquery
  • [CALCITE-751] Push aggregate with aggregate functions through join
  • Add RelBuilder.avg
  • [CALCITE-806] ROW_NUMBER should emit distinct values
  • Document JSON model, making javadoc consistent with the model reference
  • [CALCITE-808] Optimize ProjectMergeRule
  • [CALCITE-791] Optimize RelOptUtil.pushFilterPastProject
  • [CALCITE-783] Infer collation of Project using monotonicity (Milinda Pathirage)
  • Change the argument types of SqlOperator.getMonotonicity to allow it to be used for RexNode as well as SqlNode
  • [CALCITE-800] Window function defined within another window function should be invalid (Hsuan-Yi Chu)
  • [CALCITE-787] Star table wrongly assigned to materialized view (Amogh Margoor)
  • Remove duplicate resources from XML test reference files
  • [CALCITE-795] Loss of precision when sending a decimal number via the remote JSON service (Lukáš Lalinský)
  • [CALCITE-774] When GROUP BY is present, ensure that window function operands only refer to grouping keys (Hsuan-Yi Chu)
  • [CALCITE-799] Incorrect result for HAVING count(*) > 1
  • [CALCITE-801] NullPointerException using USING on table alias with column aliases
  • [CALCITE-390] Infer predicates for semi-join
  • [CALCITE-789] MetaImpl.MetaCatalog should expose TABLE_CAT instead of TABLE_CATALOG
  • [CALCITE-752] Add back sqlline as a dependency to csv example
  • [CALCITE-780] HTTP error 413 when sending a long string to the Avatica server
  • In RelBuilder, calling sort then limit has same effect as calling sortLimit
  • Add Ord.reverse
  • [CALCITE-788] Allow EnumerableJoin to be sub-classed (Li Yang)
  • [CALCITE-280] BigDecimal underflow (Li Yang)
  • [CALCITE-763] Missing translation from Sort to MutableSort (Maryann Xue)
  • [CALCITE-770] Ignore window aggregates and ranking functions when finding aggregate functions
  • [CALCITE-765] Set Content-Type from the RPC server to application/json (Lukáš Lalinský)
  • Fix Windows line-endings in RelBuilderTest
  • [CALCITE-727] Constant folding involving CASE and NULL
  • Related to [CALCITE-758], speed up matching by not considering tiles separately from other materialized views
  • Test case and workaround for [CALCITE-760] Aggregate recommender blows up if row count estimate is too high
  • [CALCITE-753] Aggregate operators may derive row types with duplicate column names
  • [CALCITE-457] Push condition of non-ansi join into join operator
  • Change jsonRequest encoding to UTF-8 (Guitao Ding)
  • [CALCITE-757] Fix expansion of view of another view (Venki Korukanti)
  • Fix coverity warnings
  • Remove deprecated SqlTypeName methods
  • [CALCITE-754] Validator error when resolving OVER clause of JOIN query
  • [CALCITE-429] Cardinality provider for use by lattice algorithm
  • [CALCITE-740] Redundant WHERE clause causes wrong result in MongoDB adapter
  • [CALCITE-665] ClassCastException in MongoDB adapter
  • Separate TableFactory from suggested table name, so one TableFactory can be used for several tables
  • [CALCITE-749] Add MaterializationService.TableFactory (Rajat Venkatesh)
  • [CALCITE-718] Enable fetch to work for Statement.execute() for Avatica (Xavier Leong)
  • [CALCITE-712] Obey setMaxRows for statement execute (Xavier Leong)
  • Add LoggingLocalJsonService, to make it easier to test that JDBC requests cause the right RPCs
  • [CALCITE-708] Implement DatabaseMetaData.getTypeInfo (Xavier Leong)
  • Enable Travis CI on new-master branch and bug-fix branches named “NNN-description”
  • Clean up
  • Upgrade tpcds
  • Make JdbcTest.testVersion more permissive, so that version.major and version.minor can be set just before a release, rather than just after as at present

1.3.0-incubating / 2015-05-30

Mainly bug-fixes, but this release adds support for modifiable views and filtered aggregate functions and various improvements to Avatica.

New features

  • [CALCITE-505] Support modifiable view
  • [CALCITE-704] FILTER clause for aggregate functions
  • [CALCITE-522] In remote JDBC driver, transmit static database properties as a map
  • [CALCITE-661] Remote fetch in Calcite JDBC driver
  • Support Date, Time, Timestamp parameters

API changes

  • [CALCITE-722] Rename markdown files to lower-case
  • [CALCITE-697] Obsolete class RelOptQuery
  • [CALCITE-693] Allow clients to control creation of RelOptCluster
  • [CALCITE-691] Allow projects to supply alternate SQL parser
  • [CALCITE-675] Enable AggregateProjectMergeRule in standard rule set
  • [CALCITE-679] Factory method for SemiJoin
  • [CALCITE-674] Add a SWAP_OUTER static instance to JoinCommuteRule (Maryann Xue)
  • [CALCITE-735] Primitive.DOUBLE.min should be large and negative

Bug-fixes and internal changes

  • [CALCITE-688] splitCondition does not behave correctly when one side of the condition references columns from different inputs
  • [CALCITE-259] Using sub-queries in CASE statement against JDBC tables generates invalid Oracle SQL (Yeong Wei)
  • In sample code in README.md, rename optiq to calcite (Ethan)
  • [CALCITE-720] VolcanoPlanner.ambitious comment doc is inconsistent (Santiago M. Mola)
  • [CALCITE-729] IndexOutOfBoundsException in ROLLUP query on JDBC data source
  • [CALCITE-733] Multiple distinct-COUNT query gives wrong results
  • [CALCITE-730] ClassCastException in table from CloneSchema
  • [CALCITE-728] Test suite hangs on Windows
  • [CALCITE-723] Document lattices
  • [CALCITE-515] Add Apache headers to markdown files
  • Upgrade quidem
  • [CALCITE-716] Scalar sub-query and aggregate function in SELECT or HAVING clause gives AssertionError
  • [CALCITE-694] Scan HAVING clause for sub-queries and IN-lists (Hsuan-Yi Chu)
  • Upgrade hydromatic-resource-maven-plugin
  • [CALCITE-710] Identical conditions in the WHERE clause cause AssertionError (Sean Hsuan-Yi Chu)
  • [CALCITE-695] Do not add SINGLE_VALUE aggregate function to a sub-query that will never return more than one row (Hsuan-Yi Chu)
  • Add tests for scalar sub-queries, including test cases for [CALCITE-709] Errors with LIMIT inside scalar sub-query
  • [CALCITE-702] Add validator test for monotonic expressions
  • [CALCITE-699] In Avatica, synchronize access to Calendar
  • [CALCITE-700] Pass time zone into tests
  • [CALCITE-698] For GROUP BY (), areColumnsUnique() should return true for any key
  • Disable tests that fail under JDK 1.7 due to [CALCITE-687]
  • Add “getting started” to HOWTO
  • [CALCITE-692] Add back sqlline as a dependency
  • [CALCITE-677] RemoteDriverTest.testTypeHandling fails east of Greenwich
  • Disable test for [CALCITE-687] Make RemoteDriverTest.testStatementLifecycle thread-safe
  • [CALCITE-686] SqlNode.unparse produces invalid SQL
  • [CALCITE-507] Update HOWTO.md with running integration tests
  • Add H2 integration test
  • Add PostgreSQL integration test
  • [CALCITE-590] Update MongoDB test suite to calcite-test-dataset
  • Add CalciteAssert.assertArrayEqual for more user-friendly asserts
  • [CALCITE-585] Avatica JDBC methods should throw SQLFeatureNotSupportedException (Ng Jiunn Jye)
  • [CALCITE-671] ByteString does not deserialize properly as a FetchRequest parameter value
  • [CALCITE-676] AssertionError in GROUPING SETS query
  • [CALCITE-678] SemiJoinRule mixes up fields when Aggregate.groupSet is not field #0

1.2.0-incubating / 2015-04-07

A short release, less than a month after 1.1.

There have been many changes to Avatica, hugely improving its coverage of the JDBC API and overall robustness. A new provider, JdbcMeta, allows you to remote an existing JDBC driver.

[CALCITE-606] improves how the planner propagates traits such as collation and distribution among relational expressions.

[CALCITE-613] and [CALCITE-307] improve implicit and explicit conversions in SQL.

New features

  • [CALCITE-366] Support Aggregate push down in bushy joins (Jesus Camacho Rodriguez)
  • [CALCITE-613] Implicitly convert character values in comparisons
  • [CALCITE-307] Implement CAST between date-time types
  • [CALCITE-634] Allow ORDER BY aggregate function in SELECT DISTINCT, provided that it occurs in SELECT clause (Sean Hsuan-Yi Chu)
  • In linq4j, implement firstOrDefault, single, and singleOrDefault methods (Daniel Cooper)
  • JDBC adapter
    • [CALCITE-631] Push theta joins down to JDBC adapter (Ng Jiunn Jye)
    • [CALCITE-657] NullPointerException when executing JdbcAggregate.implement method (Yuri Au Yong)
  • Metadata
    • [CALCITE-659] Missing types in averageTypeValueSize method in RelMdSize (Jesus Camacho Rodriguez)
    • [CALCITE-650] Add metadata for average size of a tuple in SemiJoin (Jesus Camacho Rodriguez)
    • [CALCITE-649] Extend splitCondition method in RelOptUtil to handle multiple joins on the same key (Jesus Camacho Rodriguez)

Avatica features and bug fixes

  • [CALCITE-670] AvaticaPreparedStatement should support execute() and executeUpdate() (Nick Dimiduk)
  • [CALCITE-641] Implement logging throughout Avatica server (Nick Dimiduk)
  • [CALCITE-646] AvaticaStatement.execute method broken over remote JDBC (Yeong Wei and Julian Hyde)
  • [CALCITE-660] Improve Avatica date support
  • [CALCITE-655] Implement ConnectionSync RPC (Nick Dimiduk)
  • [CALCITE-654] Tighten up AvaticaStatement.execute semantics (Nick Dimiduk)
  • [CALCITE-658] Cleanup dependency usage (Nick Dimiduk)
  • [CALCITE-652] Move server pieces of avatica into avatica-server (Nick Dimiduk)
  • [CALCITE-651] In JdbcMeta, convert property definitions to an enum (Nick Dimiduk)
  • [CALCITE-640] Avatica server should expire stale connections/statements (Nick Dimiduk)
  • [CALCITE-639] Open up permissions on avatica server components (Nick Dimiduk)
  • [CALCITE-637] Implement Avatica CloseConnection RPC (Nick Dimiduk)
  • [CALCITE-636] Connection isolation for Avatica clients (Nick Dimiduk)
  • [CALCITE-626] Implement CloseStatement RPC (Nick Dimiduk)
  • [CALCITE-630] Flesh out AvaticaParameter.setObject (Nick Dimiduk)
  • [CALCITE-627] Add Avatica support for getTableTypes, getColumns (Xavier FH Leong)
  • [CALCITE-618] Add Avatica support for getTables (Julian Hyde and Nick Dimiduk)

API changes

  • [CALCITE-617] Check at initialization time in CachingInvocationHandler that MD provider is not null (Jesus Camacho Rodriguez)
  • [CALCITE-638] SQL standard REAL is 4 bytes, FLOAT is 8 bytes

Bug-fixes and internal changes

  • [CALCITE-672] SQL ANY type should be nullable (Jinfeng Ni)
  • Disable tests, pending [CALCITE-673] Timeout executing joins against MySQL
  • Fix traits in MongoDB adapter, and NullPointerException in JdbcTest
  • [CALCITE-662] Query validation fails when an ORDER BY clause is used with WITH CLAUSE
  • [CALCITE-606] Fix trait propagation and add test case
  • Remove checkstyle Eclipse properties from git tracking
  • [CALCITE-644] Increase check style line limit to 100 chars (Nick Dimiduk)
  • [CALCITE-648] Update ProjectMergeRule description for new naming convention (Jinfeng Ni)
  • [CALCITE-625] README.md linking to the wrong page of optiq-csv (hongbin ma)
  • [CALCITE-632] Sort order returned by SUPERCLASS_COMPARATOR in ReflectiveRelMetadataProvider is inconsistent (Jesus Camacho Rodriguez)
  • [CALCITE-335] Remove uses of linq4j Functions.adapt
  • [CALCITE-592] Upgrade to Guava 14.0.1
  • [CALCITE-596] JDBC adapter incorrectly reads null values as 0 (Ng Jiunn Jye)
  • [CALCITE-633] WITH ... ORDER BY cannot find table
  • [CALCITE-614] IN clause in CASE in GROUP BY gives AssertionError
  • [CALCITE-619] Slim down dependencies in parent POM

1.1.0-incubating / 2015-03-13

This Calcite release makes it possible to exploit physical properties of relational expressions to produce more efficient plans, introducing collation and distribution as traits, Exchange relational operator, and several new forms of metadata.

We add experimental support for streaming SQL.

This release drops support for JDK 1.6; Calcite now requires 1.7 or later.

We have introduced static create methods for many sub-classes of RelNode. We strongly suggest that you use these rather than calling constructors directly.

New features

  • SQL
    • [CALCITE-602] Streaming queries (experimental)
    • [CALCITE-588] Allow TableMacro to consume maps and collections
    • [CALCITE-583] Operator || mishandles ANY type (Sean Hsuan-Yi Chu)
  • Planner rule improvements
    • [CALCITE-445] Pull up filters rejected by a ProjectableFilterableTable
    • [CALCITE-600] Use SetOpFactory in rules containing Union operator (Jesus Camacho Rodriguez)
    • [CALCITE-603] Metadata providers for size, memory, parallelism
      • [CALCITE-607] Change visibility of constructor in metadata providers for size, memory, parallelism (Jesus Camacho Rodriguez)
      • [CALCITE-608] Exception is thrown when RelMdDistribution for Project operator is called (Jesus Camacho Rodriguez)
  • Collation and distribution as traits
    • [CALCITE-88] Add collation as a trait and a kind of RelNode metadata
    • [CALCITE-569] ArrayIndexOutOfBoundsException when deducing collation (Aman Sinha)
    • [CALCITE-581] Add LogicalSort relational expression, and make Sort abstract
    • [CALCITE-526] Add EnumerableMergeJoin, which exploits sorted inputs
    • [CALCITE-71] Provide a way to declare that tables are sorted
    • [CALCITE-576] Make RelCollation trait and AbstractRelNode.getCollationList consistent
    • [CALCITE-254] Propagate RelCollation on aliased columns in JoinRule
    • [CALCITE-569] ArrayIndexOutOfBoundsException when deducing collation
    • [CALCITE-594] Add RelDistribution trait and Exchange relational expression

API changes

  • Many sub-classes of RelNode now have a static create method which automatically sets up traits such as collation and distribution. The constructors are not marked deprecated, but we strongly suggest that you use the create method if it exists.
  • [CALCITE-591] Drop support for Java 1.6 (and JDBC 4.0)
  • [CALCITE-587] Upgrade jetty-server to 9.2.7.v20150116 and port avatica-server HttpServer (Trevor Hartman)
  • [CALCITE-577] Revert temporary API changes introduced in [CALCITE-575]
  • Add means to create Context instances by wrapping objects and by chaining contexts
  • [CALCITE-599] EquiJoin in wrong package (Jesus Camacho Rodriguez)
  • [CALCITE-573] Use user-given names in RelOptUtil.createProject and createRename
  • [CALCITE-572] Remove Project.flags (methods are deprecated, to be removed before 2.0)

Bug-fixes and internal changes

  • Remove the LICENSE file of calcite-example-csv (the former optiq-csv) and move its history into main history
  • [CALCITE-615] AvaticaParameter should be Jackson serializable (Nick Dimiduk)
  • [CALCITE-612] Update AvaticaStatement to handle cancelled queries (Parth Chandra)
  • [CALCITE-605] Reduce dependency on third-party maven repositories
  • [CALCITE-611] Method setAggChildKeys should take into account indicator columns of Aggregate operator (Jesus Camacho Rodriguez)
  • [CALCITE-566] ReduceExpressionsRule requires planner to have an Executor
  • Refactor TableScanNode.create method
  • [CALCITE-593] Validator in Frameworks should expand identifiers (Jinfeng Ni)
  • Australian time-zones changed in tzdata2014f, Java 1.8.0_31
  • [CALCITE-580] Average aggregation on an Integer column throws ClassCastException
  • In Travis, ask Surefire to print results to screen
  • [CALCITE-586] Prevent JSON serialization of Signature.internalParameters

1.0.0-incubating / 2015-01-31

Calcite’s first major release.

Since the previous release we have re-organized the into the org.apache.calcite namespace. To make migration of your code easier, we have described the mapping from old to new class names as an attachment to [CALCITE-296].

The release adds SQL support for GROUPING SETS, EXTEND, UPSERT and sequences; a remote JDBC driver; improvements to the planner engine and built-in planner rules; improvements to the algorithms that implement the relational algebra, including an interpreter that can evaluate queries without compilation; and fixes about 30 bugs.

New features

  • SQL
    • [CALCITE-494] Support NEXT/CURRENT VALUE FOR syntax for using sequences
    • [CALCITE-492] Support UPSERT statement in parser
    • [CALCITE-493] Add EXTEND clause, for defining columns and their types at query/DML time
    • [CALCITE-497] Support optional qualifier for column name references
    • [CALCITE-356] Allow column references of the form schema.table.column
    • [CALCITE-462] Allow table functions in LATERAL expression
    • [CALCITE-282] Add {fn QUARTER(date)} function (Benoy Antony)
    • Grouping sets
      • [CALCITE-370] Support GROUPING SETS, CUBE, ROLLUP in SQL and algebra
      • [CALCITE-512] Add GROUP_ID,GROUPING_ID, GROUPING functions
  • Planner rule improvements
    • [CALCITE-92] Optimize away Project that merely renames fields
    • Detect and merge duplicate predicates AND(x, y, x) to AND(x, y) in more circumstances
    • [CALCITE-557] Speed up planning by never creating AbstractConverter
    • [CALCITE-545] When a projected expression can only have one value, replace with that constant
    • Grouping sets
      • [CALCITE-542] Support for Aggregate with grouping sets in RelMdColumnOrigins (Jesus Camacho Rodriguez)
      • [CALCITE-533] Support for grouping sets in FilterAggregateTransposeRule (Jesus Camacho Rodriguez)
      • [CALCITE-532] Support for grouping sets in AggregateFilterTransposeRule (Jesus Camacho Rodriguez)
      • [CALCITE-513] Support for grouping sets in AggregateProjectMergeRule (Jesus Camacho Rodriguez)
      • [CALCITE-510] Support for grouping sets in AggregateExpandDistinctAggregatesRule (Jesus Camacho Rodriguez)
      • [CALCITE-502] Support for grouping sets in AggregateUnionTransposeRule (Jesus Camacho Rodriguez)
      • [CALCITE-503] Tests to check rules on Aggregate operator without grouping sets (Jesus Camacho Rodriguez)
  • Algorithms
    • [CALCITE-451] Implement theta join, inner and outer, in enumerable convention
    • [CALCITE-489] Update Correlate mechanics and implement EnumerableCorrelate (aka nested loops join)
    • [CALCITE-544] Implement Union in interpreter
    • [CALCITE-562] Implement inner Join in interpreter and improve handling of scalar expressions
    • [CALCITE-543] Implement Aggregate (including GROUPING SETS) in interpreter (Jacques Nadeau)
    • In progress towards [CALCITE-558] add BINDABLE convention (but ENUMERABLE is still the default), and add ArrayBindable and Scalar interfaces
  • Remote driver
    • [CALCITE-93] Calcite RPC server
    • [CALCITE-94] Remote JDBC driver
    • Make JsonHandler and JsonService thread-safe

API changes

  • The great code re-org
    • [CALCITE-296] Re-organize package structure
    • [CALCITE-419] Naming convention for planner rules
    • [CALCITE-306] Standardize code style for “import package.*;”
    • [CALCITE-474] Clean up rule naming in order to support enabling/disabling rules
    • [CALCITE-460] Add ImmutableBitSet and replace uses of BitSet
    • [CALCITE-479] Migrate RelNode.getChildExps to RelNode.accept(RexShuttle)
    • [CALCITE-527] Drop rowType field and constructor/copy argument of Calc
  • Add linq4j and example-csv modules
    • Remove unused packages in linq4j, and fix checkstyle issues in linq4j and csv
    • Add calcite-linq4j and calcite-example-csv as POM sub-modules
    • Import ‘optiq-csv’ project as ‘example/csv/’, and add Apache headers
    • Import ‘linq4j’ project, and add Apache headers
    • [CALCITE-478] Move CSV tutorial (Siva Narayanan)
  • [CALCITE-464] Make parser accept configurable max length for SQL identifier
  • [CALCITE-465] Remove OneRow and Empty relational expressions; Values will suffice

Bug-fixes and internal changes

  • Build improvements
    • [CALCITE-541] Update maven-source-plugin to 2.4 to get speedup in jdk 1.8
    • [CALCITE-537] Skip overwrite of NOTICE, DEPENDENCIES, and LICENSE files
    • [CALCITE-538] Generate Parser.jj only at first build
    • [CALCITE-539] Avoid rewrite of org-apache-calcite-jdbc.properties
    • [CALCITE-540] Create git.properties file only at first build. This saves time in development at a cost of stale git.properties
    • [CALCITE-536] Add @PackageMarker to package-info.java so maven-compiler skips compilation when the sources are unchanged
    • [CALCITE-535] Support skip overwrite in hydromatic-resource
  • [CALCITE-582] EnumerableTableScan broken when table has single column
  • [CALCITE-575] Variant of ProjectRemoveRule that considers a project trivial only if its field names are identical (John Pullokkaran)
  • [CALCITE-571] ReduceExpressionsRule tries to reduce SemiJoin condition to non-equi condition
  • [CALCITE-568] Upgrade to a version of pentaho-aggdesigner that does not pull in servlet-api
  • [CALCITE-567] Make quidem dependency have scope “test”
  • [CALCITE-570] ReduceExpressionsRule throws “duplicate key” exception
  • [CALCITE-561] Upgrade parent POM
  • [CALCITE-458] ArrayIndexOutOfBoundsException when using just a single column in interpreter
  • Fix spurious extra row from FULL JOIN
  • [CALCITE-554] Outer join over NULL keys generates wrong result
  • [CALCITE-489] Teach CalciteAssert to respect multiple settings
  • [CALCITE-516] GROUP BY on a CASE expression containing IN predicate fails (Aman Sinha)
  • [CALCITE-552] Upgrade tpcds (which depends on an old version of guava)
  • Copy identifier when fully-qualifying, so column aliases have the right case
  • [CALCITE-548] Extend induce method to return CUBE and ROLLUP (Jesus Camacho Rodriguez)
    • Simplify Group.induce by assuming that group sets are sorted
  • Test case for [CALCITE-212] Join condition with OR
  • [CALCITE-550] Case-insensitive matching of sub-query columns fails
    • Add more unit tests (Jinfeng Ni)
  • [CALCITE-448] FilterIntoJoinRule creates filters containing invalid RexInputRef
  • When registering a RelNode, be tolerant if it is equivalent to a RelNode with different traits
  • [CALCITE-547] Set nullability while inferring return type of item(any,...) operator
  • In Travis CI, enable containers, and cache .m2 directory
  • [CALCITE-534] Missing implementation of ResultSetMetaData.getColumnClassName (Knut Forkalsrud)
  • [CALCITE-506] Update EnumerableRelImplementor.stash so it is suitable for all kinds of classes
  • Merge join algorithm for Enumerables
  • Efficient Enumerable over random-access list
  • Add a test that calls all functions with arguments of all types that they claim to accept
  • [CALCITE-511] copy method in LogicalAggregate not copying the indicator value properly
  • Add a model that has lattices and works against HSQLDB
  • [CALCITE-509] RelMdColumnUniqueness uses ImmutableBitSet.Builder twice, gets NullPointerException
  • [CALCITE-488] Enumerable<Holder> does not work if where Holder is a custom class with a single field; Calcite tries to treat it as SCALAR due to premature JavaRowFormat.optimize
  • [CALCITE-352] Throw exception if ResultSet.next() is called after close()
  • [CALCITE-403] Enumerable gives NullPointerException with NOT on nullable expression
  • [CALCITE-469] Update example/csv README.md instructions
  • Document WITH, LATERAL, GROUPING SETS, CUBE, ROLLUP; add descriptions for all built-in functions and operators
  • [CALCITE-470] Print warning when column type hint is not understood; Update EMPS.deptno column Integer → int
  • Fix Linq4j.product; the cartesian product of 0 attributes is one row of 0 attributes
  • Update link optiq-mat-plugin → mat-calcite-plugin
  • [CALCITE-467] Incorrect namespace in package-info.java
  • Add headers, to appease the RAT
  • [CALCITE-446] CSV adapter should read from directory relative to the model file
  • Add examples of scannable and filterable tables, matching [CALCITE-436] Simpler SPI to query Table
  • Fix JdbcTest.testVersion now that version is 1.0
  • Update release HOWTO

0.9.2-incubating / 2014-11-05

A fairly minor release, and last release before we rename all of the packages and lots of classes, in what we expect to call 1.0. If you have an existing application, it’s worth upgrading to this first, before you move on to 1.0.

New features

API changes

  • [CALCITE-447] Change semi-join rules to make use of factories
  • [CALCITE-442 Add RelOptRuleOperand constructor that takes a predicate

Bug-fixes and internal changes

  • [CALCITE-397] SELECT DISTINCT * on reflective schema gives ClassCastException at runtime
  • Various lattice improvements.
  • sqlline: Looking for class-path in inconsistent locations.
  • Re-order test suite, so that fast tests are run first.
  • [CALCITE-444] Filters wrongly pushed into full outer join
  • Make it more convenient to unit test RelMetadataQuery, and add some more tests for [CALCITE-443]
  • [CALCITE-443] getPredicates from a Union is not correct
  • Update references to web sites, git repositories, jira, mailing lists, travis CI now that [INFRA-8413] is fixed
  • [CALCITE-435] FilterAggregateTransposeRule loses conditions that cannot be pushed
  • [CALCITE-435] LoptOptimizeJoinRule incorrectly re-orders outer joins
  • [CALCITE-439] SqlValidatorUtil.uniquify() may not terminate under some conditions
  • [CALCITE-438] Push predicates through SemiJoinRel
  • Add test case for LIKE ... ESCAPE.
  • HOWTO: Modify release instructions.
  • Update DiffRepository documentation.
  • Add tests for windowed aggregates without ORDER BY. (Works already.)

0.9.1-incubating / 2014-10-02

This is the first release as Calcite. (The project was previously called Optiq.)

New features

  • [CALCITE-430] Rename project from Optiq to Calcite
  • [CALCITE-426] Pool JDBC data sources, to make it easier to pool connections
  • [CALCITE-416] Execute logical RelNodes using an interpreter
  • [CALCITE-376] Move SqlRun into its own artifact, Quidem.
  • [CALCITE-269] MongoDB result sets larger than 16MB
  • [CALCITE-373] NULL values in NOT IN sub-queries
  • SQL functions:
    • [CALCITE-422] Add REGR_SXX and REGR_SYY regression functions
    • [CALCITE-421] Add COVAR_POP and COVAR_SAMP aggregate functions
  • Planner rules:
    • [CALCITE-425] Add FilterAggregateTransposeRule, that pushes a filter through an aggregate
    • [CALCITE-399] Factorize common AND factors out of OR predicates
    • [CALCITE-404] MergeProjectRule should not construct RexPrograms for simple mappings
    • [CALCITE-394] Add RexUtil.toCnf(), to convert expressions to conjunctive normal form (CNF)
    • [CALCITE-389] MergeFilterRule should flatten AND condition
  • Lattices:
    • [CALCITE-428] Use optimization algorithm to suggest which tiles of a lattice to materialize
    • [CALCITE-410] Allow lattice tiles to satisfy a query by rolling up
    • [CALCITE-406] Add tile and measure elements to lattice model element
    • Now, a lattice can materialize an aggregate-join and use it in a subsequent query.
    • [CALCITE-402] Lattice should create materializations on demand
    • [CALCITE-344] Lattice data structure
  • Field trimmer:
    • [CALCITE-408] Make FieldTrimmer work with RelNode base classes
    • [CALCITE-388] Handle semi-joins in field trimmer
    • [CALCITE-395] Make FieldTrimmer.trimFields(SetOp) generate ProjectRel instead of CalcRel
    • [CALCITE-393] If no fields are projected from a table, field trimmer should project a dummy expression

API changes

  • [CALCITE-413] Add RelDataTypeSystem plugin, allowing different max precision of a DECIMAL
  • In Planner, query de-correlation no longer requires state in a SqlToRelConverter.
  • Factories:
    • [CALCITE-392] RelFieldTrimmer should use factory to create new rel nodes
    • [CALCITE-382] Refactoring rules to use factories
    • [CALCITE-398] Move CalcRel.createProject methods to RelOptUtil
    • [CALCITE-396] Change return type of JoinFactory.createJoin(); add SemiJoinFactory

Bug-fixes and internal changes

  • [CALCITE-386] Fix NOTICE
  • Add tests inspired by Derby bugs.
  • Add recent presentation to README.md.
  • [CALCITE-427] Off-by-one issues in RemoveDistinctAggregateRule, AggregateFilterTransposeRule
  • [CALCITE-414] Bad class name in sqlline shell script
  • Bad package name in package-info.java was causing errors in Eclipse.
  • [CALCITE-412] RelFieldTrimmer: when trimming SortRel, the collation and trait set don’t match
  • Add test case for [CALCITE-411] Duplicate column aliases
  • [CALCITE-407] RemoveTrivialProjectRule drops child node’s traits
  • [CALCITE-409] PushFilterPastProjectRule should not push filters past windowed aggregates
  • Fix tests on Windows.
  • Don’t load FoodMartQuerySet unless we have to. It’s big.
  • Enable connection pooling in test suite.
  • [CALCITE-384] Add apache- prefix to tarball and directory within tarball
  • Freeze hive fmpp > freemarker plugin dependency.
  • Upgrade Janino
  • Removed hardcoded foodmart schema information
  • [CALCITE-387] CompileException when cast TRUE to nullable boolean
  • Temporary fix for [CALCITE-390] Transitive inference (RelMdPredicates) doesn’t handle semi-join
  • [CALCITE-385] Change comment style for Java headers
  • Disable test that is inconistent between JDK 1.7 and 1.8.
  • Fix git-commit-id-plugin error when running in Travis-CI.
  • [CALCITE-381] Remove plugin versions from the <plugins> tag in root pom
  • [CALCITE-383] Each jar should have a git.properties file describing its exact version
  • Fix mvn site on JDK 1.8 and enable in Travis-CI.
  • Status icon based on master branch, not whichever branch happened to build most recently.
  • HOWTO:
    • Document how to build from git, and how to get jars from maven repo.
    • Optiq web site
    • Template emails for Apache votes
    • Update JIRA cases following release
    • Instructions for making and verifying a release

0.9.0-incubating / 2014-08-19

This is the first release under the Apache incubator process.

New features

  • [CALCITE-371] Implement JOIN whose ON clause contains mixed equi and theta
  • [CALCITE-369] Add EnumerableSemiJoinRel, implementation of semi-join in enumerable convention
  • Add class Strong, for detecting null-rejecting predicates.
  • [CALCITE-368] Add SemiJoinRule, planner rule to convert project-join-aggregate into semi-join
  • [CALCITE-367] PushFilterPastJoinRule should strengthen join type
  • Add EquiJoinRel, base class for joins known to be equi-joins.
  • Implement CAST(<string> AS <datetime>) and <datetime> + <interval>.
  • [CALCITE-360] Introduce a rule to infer predicates from equi-join conditions
  • [CALCITE-349] Add heuristic join-optimizer that can generate bushy joins
  • [CALCITE-346] Add commutative join rule
  • [CALCITE-347] In SqlRun, add !plan command
  • [CALCITE-314] Allow simple UDFs based on methods
  • [CALCITE-327] Rules should use base class to find rule match & use factory for object creation
  • [CALCITE-316] In SqlRun, match output regardless of order if ORDER BY not present
  • [CALCITE-300] Support multiple parameters in COUNT(DISTINCT x, y, ...)

API changes

  • [CALCITE-343] RelDecorrelator should build its own mappings, not inherit from SqlToRelConverter
  • Remove deprecated methods.
  • Convert Hook to use Guava Function (was linq4j Function1).
  • Add fluent method withHook, to more easily add hooks in tests.
  • [CALCITE-321] Add support for overriding implementation of CompoundIdentifier in SqlParser.
  • [CALCITE-322] Add support for SqlExplain, SqlOrderBy and SqlWith to support SqlShuttle use.
  • [CALCITE-323] Override SqlUnresolvedFunction.inferReturnType() to return ANY type so framework implementors can support late bound function implementations.
  • [CALCITE-324] Add ViewExpander for Planner in Frameworks. Expose additional properties of ViewTable to allow subclassing.
  • [CALCITE-247] Add Context and FrameworkConfig

Bug-fixes and internal changes

  • [CALCITE-380] Downgrade to Guava 11.0.2
  • Move several .md files into new ‘doc’ directory, to keep the root directory simple.
  • Add DISCLAIMER
  • Update history and HOWTO
  • [CALCITE-377] UnregisteredDriver should catch, log and re-throw NoClassDefFoundError
  • Inherit maven-release-plugin from Apache POM.
  • Test case for [CALCITE-373] NOT IN and NULL values
  • [CALCITE-372] Change LoptOptimizeJoinRule & PushFilterPast* rules to use factory
  • Upgrade maven-checkstyle-plugin.
  • Add class Holder, a mutable slot that can contain one object.
  • Remove the 2-minute wait at the top of the hour for tests of CURRENT_TIME, etc.
  • Tune ImmutableIntList’s iterators.
  • [CALCITE-364] Validator rejects valid WITH ... ORDER BY query
  • [CALCITE-363] Use dependencyManagement and pluginManagement in POM files
  • Add FilterFactory.
  • Add README file, incubation disclaimers, and how-to build and running tests.
  • Add KEYS and start how-to for making snapshots and releases.
  • Capital case component names; inherit license info from Apache parent POM.
  • Only run apache-rat and git-commit-id-plugin in “release” maven profile.
  • [CALCITE-348] Add Apache RAT as maven plugin
  • Change license headers from “Julian Hyde” to “ASF”; add headers where missing.
  • Fix build breakage on JDK 1.6 due to missing method BitSet.previousClearBit.
  • Refactor test infrastructure to allow testing against heuristic bushy-join optimizer.
  • Add methods and tests for BitSets, and re-organize tests.
  • [CALCITE-354] Change maven groupId to “org.apache.optiq”
  • Specify return type when calling RexBuilder.makeCall, if possible.
  • Eliminate duplicate conditions in RexProgramBuilder.addCondition, not RexBuilder.makeCall as previously.
  • [CALCITE-345] AssertionError in RexToLixTranslator comparing to date literal
  • Restore PushFilterPastJoinRule to RelDecorrelator; interim pending [CALCITE-343] fix.
  • [CALCITE-340] Fix bug in SqlToRelConverter when push expressions in join conditions into ProjectRel.
  • [CALCITE-313] Query decorrelation fails
  • While unifying a RelNode tree with a materialized view expression, switch representation to MutableRels.
  • [CALCITE-305] Unit test failure on release candidates
  • [CALCITE-325] Use Java list instead of Guava list to avoid null checks in case of SqlTypeExplicitPrecedenceList.
  • [CALCITE-326] Fix RelOptUtil ANY type check.
  • [CALCITE-303] Migrate issue URLs
  • [CALCITE-331] Precision/scale compatibility checks should always succeed for ANY type
  • In SqlRun, allow !plan after !ok for same SQL statement.
  • [CALCITE-318] Add unit test for SqlRun
  • Fix a bug where composite SELECT DISTINCT would return duplicate rows.

0.8 / 2014-06-27

New features

  • [CALCITE-310] Implement LEAD, LAG and NTILE windowed aggregates
  • Reduce COUNT(not-null-expression) to COUNT()
  • [CALCITE-292] Improve windowed aggregate return types
  • [CALCITE-302] Use heuristic rule to plan queries with large numbers of joins
  • [CALCITE-283] Add TPC-DS data generator
  • [CALCITE-294] Implement DENSE_RANK windowed aggregate function
  • SqlRun utility
    • [CALCITE-290] Add SqlRun, an idempotent utility for running SQL test scripts
    • Add “!skip” command to SqlRun.
    • Add MySQL formatting mode to SqlRun.

API changes

  • Re-organize planner initialization, to make it easier to use heuristic join order.
  • [CALCITE-301] Add Program interface, a planner phase more general than current RuleSet
  • [CALCITE-263] Add operand type that will cause a rule to fire when a new subset is created
  • Clean up and document SqlKind.
    • Add IS_NOT_TRUE and IS_NOT_FALSE SqlKind enums.
    • Add SqlKind.IS_NOT_NULL enum value, and use where possible, including for IS_NOT_UNKNOWN operator.

Bug-fixes and internal changes

  • [CALCITE-312] Trim non-required fields before WindowRel
  • [CALCITE-311] Wrong results when filtering the results of windowed aggregation
  • More tests for WITH ... ORDER BY
  • [CALCITE-309] WITH ... ORDER BY query gives AssertionError
  • Enable MultiJoinRel and some other planner rule tests.
  • Add ImmutableNullableList and UnmodifiableArrayList, and remove calls to Arrays.asList.
  • Add method IntPair.zip.
  • Reimplement regular and windowed aggregates
  • Switch from github to Apache JIRA for issues tracking.
    • In release history, update issue URLs from github to Apache JIRA
  • The Apache mailing list is now the official mailing list. Add presentations.
  • Add test for overloaded UDF.
  • Add tests for NOT IN where sub-query returns NULL values.
  • [CALCITE-288] Add tests for windowed aggregation based on Postgres reference queries
  • [CALCITE-286] Error casting MongoDB date
  • [CALCITE-284] Window functions range defaults to CURRENT ROW
  • [CALCITE-285] Window functions throw exception without ORDER BY
  • Test case for [<a href=“https://issues.apache.org/jira/browse/CALCITE-285”>CALCITE-285</a>].
  • [CALCITE-281] EXTRACT function’s SQL return type is BIGINT but implemented as Java int

0.7 / 2014-05-13

New features

  • Implement table functions.
  • Arrays and multi-sets:
    • [CALCITE-267] Improve support for ARRAY data type
    • Better type information for JDBC Array; nested array now possible.
    • Implement JOIN LATERAL and JOIN UNNEST.
    • Implement the UNNEST relational operator, and various improvements to ARRAY and MULTISET data types.
    • Represent ARRAY columns as Java lists.
    • Implement CARDINALITY(ARRAY) SQL operator.
  • Implement scalar sub-query in SELECT clause.
  • [CALCITE-273] Support column alias in WITH queries (common table expressions)
  • Windowed aggregates:
    • Aggregate over constants, e.g. SUM(1) OVER (ROWS 10 PRECEDING);
    • UNBOUNDED PRECEDING window range;
    • Windowed aggregates computed over primitive scalars.
  • Fix return type inference for aggregate calls. If the GROUP BY clause is empty, SUM may return null.
  • [CALCITE-37] Document JSON model file format (as model.md).
  • [CALCITE-238] Add adapter that generates TPC-H data
  • Improve exception message in AvaticaConnection; add ExceptionMessageTest.
  • Implement micro-benchmarks via JMH.

API changes

  • Provide an option to create root schema without the “metadata” schema.
  • Schema SPI:
    • [CALCITE-175] Modify Schema SPI to allow caching
    • Get sub-schemas defined by a Schema SPI, and cache their OptiqSchema wrappers. (Tobi Vollebregt and Julian Hyde)
  • SqlAdvisor callable from client via JDBC.

Bug-fixes and internal changes

  • Add Apache incubator proposal.
  • Rename RELEASE.md to HISTORY.md.
  • Upgrade maven-release-plugin.
  • Upgrade to linq4j-0.3.
  • Code generation improvements:
  • Move code-generation optimizer to linq4j;
  • Improve translation of strict functions;
  • Mark most methods in SqlFunctions as @Deterministic;
  • Support static final constants generated by linq4j.
  • Avoid excessive box and unbox of primitives when using Object[] storage.
  • In JDBC result set, avoid row computation on each accessor call.
  • Test composite join conditions in various flavors of outer join.
  • Use fromTrait of the just previously converted RelNode instead of the original RelNode.
  • Disable a MongoDB test, pending [CALCITE-270].
  • Hush warnings from SplunkAdapterTest if Splunk is not available.
  • [CALCITE-252] Scalar sub-query that returns 0 rows should become NULL value
  • SplunkAdapterTest now uses the same Foodmart database as JdbcTest.
  • [CALCITE-242] SplunkAdapterTest fails
  • Remove some obsolete classes.
  • [CALCITE-205] Suspicious map.get in VolcanoPlanner.reregister

0.6 / 2014-04-11

New features

  • [CALCITE-214] Modify Frameworks to allow Schema to be re-used Obsoletes name field of ReflectiveSchema.
  • [CALCITE-237] Allow user-defined aggregate functions (UDAs) to be defined in a model
  • [CALCITE-227] Extend EXTRACT function to support DATE, TIME and TIMESTAMP values
  • [CALCITE-222] User-defined table macros
  • [CALCITE-179] Optiq on Windows
    • Add sqlline.bat and fix issues running sqlline under Cygwin.
  • [CALCITE-195] Push aggregation into MongoDB adapter
  • [CALCITE-193] Implement OFFSET and LIMIT in MongoDB adapter
  • [CALCITE-164] Improve query performance of optiq over MongoDB
  • Add Phoenix (HBase) SQL dialect (Bruno Dumon)

API changes

  • Obsolete RexImpTable.AggregateImplementor and rename AggImplementor2. (This is a breaking change.)
  • Convert CombinedParser.jj into freemarker template to allow custom parser implementations. (Venki Korukanti)
  • Extend Planner to pass a custom ConvertletTable and custom SQL parser.
  • In Frameworks, add a way to specify list of TraitDefs that will be used by planner. (Jinfeng Ni)
  • [CALCITE-198] Use RexExecutor to evaluate projections and filters
  • [CALCITE-219] Parse ALTER scope SET option = value statement
  • [CALCITE-215] A Schema should not have to remember its name and parent (This is a breaking change.)
  • [CALCITE-180] Common base class for TableFunction, ScalarFunction (This is a breaking change.)
  • Add methods for dealing with symbols; deprecate SqlLiteral.booleanValue(SqlNode), SqlLiteral.symbolValue(SqlNode).
  • Add RelOptPlanner.clear(); now it is safe to call transform twice. (Jinfeng Ni)
  • Remove APIs deprecated for 0.5.
  • Move around some operator classes and singletons.

Bug-fixes and internal changes

  • Upgrade to linq4j-0.2.
  • FETCH and LIMIT are ignored during SQL-to-RelNode translation. (Venki Korukanti)
  • [CALCITE-245] Off-by-one translation of ON clause of JOIN
  • [CALCITE-191] Rotate time/date/timestamp vals to local timezone
  • [CALCITE-244] RelOptTableImpl.create always expects QueryableTable type in OptiqSchema.TableEntry
  • [CALCITE-225] Optiq doesn’t correctly decorrelate queries
  • Clean up package-info. Remove duplicates in test packages so they don’t conflict with those in non-test packages.
  • Add Pair.adjacents(Iterable).
  • [CALCITE-199] Various ANY type conditions aren’t correctly being considered (Jacques Nadeau)
  • Add files to .gitignore that shouldn’t be checked in when using Eclipse. (Jacques Nadeau)
  • Add class ControlFlowException, and make it base class of existing control-flow exception classes.
  • [CALCITE-232] Sum and avg of empty set should be null as per SQL specification
  • Add SqlUnresolvedFunction, to improve how return type of user-defined functions is resolved. (Vladimir Sitnikov)
  • [CALCITE-228] Error while compiling generated Java code when using UDF in expression
  • [CALCITE-226] User-defined functions should work without explicit schema prefix
  • [CALCITE-229] Join between different JDBC schemas not implementable
  • [CALCITE-230] RemoveSortRule derives trait set from sort, should derive it from sort’s child
  • Test view and sub-query with ORDER BY and LIMIT.
  • [CALCITE-223] Add NOTICE and LICENSE files in all generated JAR files
  • [CALCITE-209] Consistent strategy for line-endings in tests Convert uses of NL in tests to Linux newline “\n”. This makes string constants simpler.
  • [CALCITE-218] Functions case sensitive when using Lex.MYSQL
  • Add tests that a query with aggregate expressions in the SELECT clause is considered an aggregate query, even if there is no GROUP BY.
  • [CALCITE-216] Inconsistent use of provided operator table causes inability to add aggregate functions
  • [CALCITE-200] Javadoc generation fails under JDK 1.8
  • Add class XmlOutput (based on org.eigenbase.xom.XMLOutput) and remove dependency on eigenbase-xom.
  • Performance: Don’t create stack-trace for exceptions used for control-flow. (Vladimir Sitnikov)
  • Performance: Tune RexProgramBuilder by using Pair rather than String as expression key. (Vladimir Sitnikov)
  • Fix NPE using TRIM function with JDBC. (Bruno Dumon)
  • Add dependency on hydromatic-resource-maven-plugin and obsolete our copy of the resource framework.
  • Fix race condition in SpaceList.
  • In planner, use RelTrait.subsumes rather than equals in an assert. (Jinfeng Ni)

0.5 / 2014-03-14

New features

  • Allow quoting, quotedCasing, unquotedCasing, and caseSensitive properties to be specified explicitly (Vladimir Sitnikov)
  • Recognize more kinds of materializations, including filter-on-project (where project contains expressions) and some kinds of aggregation.
  • [CALCITE-128] Support WITH queries (common table expressions)
  • [CALCITE-53] Allow WHEN clause in simple CASE expression to have multiple values
  • [CALCITE-156] Optiq should recognize ‘SYSTEM TABLE’, ‘JOIN’, ‘INDEX’ as table types
  • Support querying ARRAY columns from JDBC source. (Gabriel Reid)

API changes

  • Add ProjectRelBase.copy(RelTraitSet, RelNode, List<RexNode>, RelDataType) and make ProjectRelBase.copy(RelTraitSet, RelNode) final. (This is a breaking change for sub-classes of ProjectRelBase.)
  • Change RexBuilder.makeRangeReference parameter type.
  • RexBuilder.makeInputRef replaces RelOptUtil.createInputRef.
  • [CALCITE-160] Allow comments in schema definitions
  • [CALCITE-147] Create a new kind of SqlCall that keeps operands in fields, not an operands array
    • Very widely used parse tree nodes with complex operands, including SqlSelect, SqlJoin, SqlInsert, and a new node type SqlOrderBy, are now sub-classes of SqlCall but not SqlBasicCall.
    • (This is a breaking change to code that assumes that, say, SqlSelect has an operands field.)
  • Convert all enum constants to upper-case. (This is a breaking change.)

Bug-fixes and internal changes

  • Generate optiq-core-VERSION-tests.jar not parent-VERSION-tests.jar.
  • [CALCITE-176] ORDER BY expression doesn’t work with SELECT *
  • Fix VARCHAR casts sent to hsqldb source (Bruno Dumon)
  • [CALCITE-143] Remove dependency on eigenbase-resgen
  • [CALCITE-173] Case-insensitive table names are not supported for Casing.UNCHANGED
  • DATE.getLimit now returns Calendar in GMT time zone (Vladimir Sitnikov)
  • Set en_US locale in tests that match against error numbers, dates (Vladimir Sitnikov)
  • Use 1 test thread per CPU to avoid thread starvation on dual core CPUs (Vladimir Sitnikov)
  • [CALCITE-174] Move hsqldb to test scope
  • Add unit tests for RexExecutorImpl.
  • Correct JSON model examples in Javadoc comments. (Karel Vervaeke)
  • Move test reference logs from src/test/java to src/test/resources (reduces the number of ‘untracked files’ reported by git)
  • Tune Util.SpaceList, fix race condition, and move into new utility class Spaces.
  • [CALCITE-163] Equi-join warning
  • [CALCITE-157] Handle SQLFeatureNotSupported when calling setQueryTimeout (Karel Vervaeke)
  • Fix Optiq on Windows. (All tests and checkstyle checks pass.)
  • In checkstyle, support Windows-style file separator, otherwise build fails in Windows due to suppressions not used. (Vladimir Sitnikov)
  • Enable MongoDB tests when -Dcalcite.test.mongodb=true.
  • Cleanup cache exception-handling and an assert.
  • [CALCITE-153] Error using MongoDB adapter: Failed to set setXIncludeAware(true)
  • Disable spark engine unless Spark libraries are on the class path and spark=true is specified in the connect string.
  • Fix path to mongo-zips-model.json in HOWTO. (Mariano Luna)
  • Fix bug deriving the type of a join-key.
  • Fix the value of ONE_MINUS_EPSILON.
  • [CALCITE-158] Optiq fails when call Planner.transform() multiple times, each with different ruleset
  • [CALCITE-148] Less verbose description of collation. Also, optimize RelTraitSet creation and amortize RelTraitSet.toString().
  • Add generics to SQL parser.
  • [CALCITE-145] Unexpected upper-casing of keywords when using java lexer
  • Remove duplicate maven-source-plugin.
  • [CALCITE-141] Downgrade to guava-11.0.2. (This is necessary for Hadoop compatibility. Later versions of Guava can also be used.)
  • Upgrade to spark-0.9.0. (Because this version of spark is available from maven-central, we can make optiq-spark part of the regular build, and remove the spark profile.)

0.4.18 / 2014-02-14

API and functionality changes

  • Configurable lexical policy
    • [CALCITE-33] SQL parser should allow different identifier quoting
    • [CALCITE-34] Policy for case-sensitivity of identifiers should be configurable
    • New connect-string parameter “lex”, with allowable values “ORACLE”, “MYSQL”, “SQL_SERVER”, “JAVA” sets policy to be like those databases, in terms of quote string, whether quoted and unquoted identifiers are converted to upper/lower case, and whether identifiers are matched case-sensitively. “JAVA” is case-sensitive, even for unquoted identifiers. It should be possible for each connection to have its own settings for these. Objects shared between sessions (views, materialized views) might require more work.
    • Added various internals to make it easy for developers to do the right thing. When you need to look up a schema, table or column/field name, you should use a catalog reader, and it will apply the right case-sensitivity policy.
    • Enable optiq consumer to utilize different lexical settings in Frameworks/Planner. (Jacques Nadeau)
  • [CALCITE-115] Add a PARSE_TREE hook point with SqlNode parameter
  • Change planner rules to use ProjectFactory for creating projects. (John Pullokkaran)
  • [CALCITE-131] Add interfaces for metadata (statistics) (This is a breaking change.)
  • Update Avatica to allow Cursor & Accessor implementations to throw SQLException. (Jacques Nadeau)
  • Separate cost model (RelOptCostFactory) from planner. Allow VolcanoPlanner to be sub-classed with different cost factory.
    • Remove references to VolcanoCost from RelSubset, so clients can use a different RelOptCost. (Harish Butani)
    • Make VolcanoCost immutable.
  • Break SqlTypeStrategies into OperandTypes, ReturnTypes and InferTypes, and rename its static members to upper-case, per checkstyle. (This is a breaking change.)
  • Add a mechanism for defining configuration parameters and have them appear in the responses to AvaticaDatabaseMetaData methods.
  • [CALCITE-113] User-defined scalar functions
  • Add rules to short-cut a query if LIMIT 0 is present. Also remove sort, aggregation, join if their inputs are known to be empty, and propagate the fact that the relational expressions are known to be empty up the tree. (We already do this for union, filter, project.)
  • RexNode and its sub-classes are now immutable.

Bug-fixes and internal changes

  • [CALCITE-16] Upgrade to janino-2.7
  • Upgrade to guava-15.0 (guava-14.0.1 still allowed), sqlline-1.1.7, maven-surefire-plugin-2.16, linq4j-0.1.13.
  • [CALCITE-136] Support Hive dialect
  • [CALCITE-138] SqlDataTypeSpec.clone handles collection types wrong
  • [CALCITE-137] If a subset is created that is subsumed by an existing subset, its ‘best’ is not assigned
    • If best rel in a Volcano subset doesn’t have metadata, see if other rels have metadata.
  • [CALCITE-127] EnumerableCalcRel can’t support 3+ AND conditions (Harish Butani)
  • Fix push-down of datetime literals to JDBC data sources.
  • Add Util.startsWith(List, List) and Util.hashCode(double).
  • Add maven-checkstyle-plugin, enable in “verify” phase, and fix exceptions.
  • Fix SqlValidator to rely on RelDataType to do field name matching. Fix RelDataTypeImpl to correctly use the case sensitive flag rather than ignoring it.
  • [CALCITE-119] Comparing Java type long with SQL type INTEGER gives wrong answer
  • Enable multi-threaded testing, and fix race conditions.
    • Two of the race conditions involved involving trait caches. The other was indeterminacy in type system when precision was not specified but had a default; now we canonize TIME to TIME(0), for instance.
  • Convert files to us-ascii.
  • Work around [JANINO-169].
  • Refactor SQL validator testing infrastructure so SQL parser is configurable.
  • Add optiq-mat-plugin to README.
  • Fix the check for duplicate subsets in a rule match.
  • [CALCITE-112] Java boolean column should be treated as SQL boolean
  • Fix escaped unicode characters above 0x8000. Add tests for unicode strings.

0.4.17 / 2014-01-13

API changes

  • [CALCITE-106] Make Schema and Table SPIs simpler to implement, and make them re-usable across connections (This is a breaking change.)
  • Make it easier to define sub-classes of rule operands. The new class RelOptRuleOperandChildren contains the children of an operand and the policy for dealing with them. Existing rules now use the new methods to construct operands: operand(), leaf(), any(), none(), unordered(). The previous methods are now deprecated and will be removed before 0.4.18. (This is a breaking change.)
  • [CALCITE-101] Enable phased access to the Optiq engine
  • List-handling methods in Util: add methods skipLast, last, skip; remove subList, butLast.
  • Convert SqlIdentifier.names from String[] to ImmutableList<String>.
  • Rename OptiqAssert.assertThat() to that(), to avoid clash with junit’s Assert.assertThat().
  • Usability improvements for RelDataTypeFactory.FieldInfoBuilder. It now has a type-factory, so you can just call build().
  • Rework HepProgramBuilder into a fluent API.
  • [CALCITE-105] Externalize RelNode to and from JSON

Tuning

  • If EnumerableAggregateRel has no aggregate functions, generate a call to Enumerable.distinct(), thereby saving the effort of building trivial accumulators.
  • Default rule set now does not introduce CalcRel until a later phase of planning. This reduces the number of trivial projects and calcs created, merged, and elimated.
  • Reduce the amount of time spent creating record types that already exist.
  • More efficient implementation of Util.isDistinct for small lists.
  • When an internal record has 0 fields, rather than generating a synthetic class and lots of instances that are all the same, use the new Unit class, which is a singleton.
  • To take advantage of asymmetric hash join added recently in linq4j, tweak cost of EnumerableJoinRel so that join is cheaper if the larger input is on the left, and more expensive if it is a cartesian product.
  • [CALCITE-70] Joins seem to be very expensive in memory
  • Make planning process more efficient by not sorting the list of matched rules each cycle. It is sorted if tracing is enabled; otherwise we scan to find the most important element. For this list, replace LinkedList with ChunkList, which has an O(1) remove and add, a fast O(n) get, and fast scan.

Other

  • [CALCITE-87] Constant folding
    • Rules for constant-expression reduction, and to simplify/eliminate VALUES operator.
  • Graph algorithms: Implement breadth-first iterator and cycle-detector.
  • Fix bug in planner which occurred when two RelNodes have identical digest but different row-type.
  • Fix link to optiq-csv tutorial.
  • Fix bugs in RemoveTrivialProjectRule.strip, JdbcProjectRel.implement and SortRel.computeSelfCost.
  • Reformat code, and remove @author tags.
  • Upgrade to eigenbase-xom-1.3.4, eigenbase-properties-1.1.4, eigenbase-resgen-1.3.6.
  • Upgrade to linq4j-0.1.12.
  • [CALCITE-97] Correlated EXISTS
  • Fix a bug in VolcanoCost.
  • Add class FoodMartQuerySet, that contains the 6,700 foodmart queries.
  • Fix factory class names in UnregisteredDriver
  • [CALCITE-96] LIMIT against a table in a clone schema causes UnsupportedOperationException
  • Disable spark module by default.
  • Allow CloneSchema to be specified in terms of url, driver, user, password; not just dataSource.
  • Wrap internal error in SQLException.

0.4.16 / 2013-11-24

  • [CALCITE-69] Can’t join on string columns and other problems with expressions in the join condition
  • [CALCITE-74] JOIN … USING fails in 3-way join with UnsupportedOperationException.
  • [CALCITE-65] Fix issues in the JDBC driver, and in particular to DatabaseMetaData methods, to make Squirrel-SQL run better.
  • Fix JDBC column, table, schema names for when the table is not in a schema of depth 1.
  • [CALCITE-85] Adding a table to the root schema causes breakage in OptiqPrepareImpl
  • [CALCITE-84] Extract Optiq’s JDBC driver as a new JDBC driver framework, Avatica. Other projects can use this to implement a JDBC driver by implementing just a few methods. If you wish to use Optiq’s JDBC driver, you will now need to include optiq-avatica.jar in addition to optiq-core.jar. Avatica does not depend on anything besides the standard Java library.
  • Support for parameters in PreparedStatement.
  • First steps in recognizing complex materializations. Internally we introduce a concept called a “star table”, virtual table composed of real tables joined together via many-to-one relationships. The queries that define materializations and end-user queries are canonized in terms of star tables. Matching (not done yet) will then be a matter of looking for sort, groupBy, project. It is not yet possible to define a star in an Optiq model file.
  • Add section to HOWTO on implementing adapters.
  • Fix data type conversions when creating a clone table in memory.
  • Fix how strings are escaped in JsonBuilder.
  • Test suite now depends on an embedded hsqldb database, so you can run mvn test right after pulling from git. You can instead use a MySQL database if you specify ‘-Dcalcite.test.db=mysql’, but you need to manually populate it.
  • Fix a planner issue which occurs when the left and right children of join are the same relational expression, caused by a self-join query.
  • [CALCITE-76] Precedence of the item operator, map[index]; remove the space before ‘[’ when converting parse tree to string.
  • Allow CAST(expression AS ANY), and fix an issue with the ANY type and NULL values.
  • Handle null timestamps and dates coming out of JDBC adapter.
  • Add jdbcDriver attribute to JDBC schema in model, for drivers that do not auto-register.
  • Allow join rules to match any subclass of JoinRelBase.
  • Push projects, filters and sorts down to MongoDB. (Fixes [CALCITE-57], [CALCITE-60] and [CALCITE-72].)
  • Add instructions for loading FoodMart data set into MongoDB, and how to enable tracing.
  • Now runs on JDK 1.8 (still runs on JDK 1.6 and JDK 1.7).
  • Upgrade to junit-4.11 (avoiding the dodgy junit-4.1.12).
  • Upgrade to linq4j-0.1.11.

0.4.15 / 2013-10-14

Optiq-csv release history

Optiq-csv-0.3 was the last independent release of optiq-csv. From calcite-0.9.2 onwards, the code was included as the calcite-example-csv module.

  • Upgrade to calcite-0.9.1
  • Support gzip-compressed CSV and JSON files (recognized by ‘.gz’ suffix)
  • Cleanup, and fix minor timezone issue in a test
  • Support for date types (date, time, timestamp) (Martijn van den Broek)
  • Upgrade to optiq-0.8, optiq-avatica-0.8, linq4j-0.4
  • Add support for JSON files (recognized by ‘.json’ suffix)
  • Upgrade maven-release-plugin to version 2.4.2
  • Upgrade to optiq-0.6, linq4j-0.2
  • Add NOTICE and LICENSE files in generated JAR file

0.3 / 2014-03-21

  • Upgrade to optiq-0.5
  • Add workaround to [JLINE2-62] to sqlline.bat (windows) and sqlline (windows using cygwin)
  • Fix classpath construction: sqlline.bat copies dependencies to target/dependencies; sqlline constructs target/classpath.txt
  • Build, checkstyle and tests now succeed on windows (both native and cygwin)
  • Models can now contain comments
  • [OPTIQ-CSV-2] Update tutorial to reflect changes to Optiq’s JDBC adapter

0.2 / 2014-02-18

  • Add test case for [CALCITE-112]
  • Add sqlline.bat, Windows SQL shell (based on fix for [DRILL-338])
  • Upgrade to optiq-0.4.18, sqlline-1.1.7
  • Return a single object for single-col enumerator (Gabriel Reid)
  • Enable maven-checkstyle-plugin; fix checkstyle exceptions

0.1 / 2014-01-13

  • Add release notes and history
  • Enable maven-release-plugin
  • Upgrade to optiq-0.4.17, linq4j-0.1.12, sqlline-1.1.6
  • Upgrade tutorial for new Schema and Table SPIs
  • Fixes for optiq SPI changes in [CALCITE-106]
  • Enable oraclejdk8 in Travis CI
  • Fix bug where non-existent directory would give NPE; instead print warning
  • Add an example of a planner rule
  • Add CsvTableFactory, an example of a custom table
  • Add a view to tutorial
  • Split into scenario with a “simple” schema that generates tables (CsvTable) that just execute and a “smart” schema that generates tables (CsvSmartTable) that undergo optimization
  • Make CsvEnumerator a top-level class
  • Implement the algorithms to sniff names and types from the first row, and to return an enumerator of all rows
  • Read column types from header of CSV file

Linq4j release history

Linq4j-0.4 was the last independent release of linq4j. From calcite-0.9.2 onwards, the code was included as calcite-linq4j, and features added to linq4j in a particular calcite release are described with the other changes in that release.

0.4 / 2014-05-28

  • Fix #27, “Incorrectly inlines non-final variable”.
  • Maven build process now deploys web site.
  • Implement Enumerable methods: any, all, contains with EqualityComparer, first, first with predicate.

0.3 / 2014-04-21

  • Move optimizer visitor from optiq to linq4j; add ExpressionType.modifiesLvalue to avoid invalid inlining.
  • Fix #17, “Assign constant expressions to ‘static final’ members”; add @Deterministic annotation to help deduce which expressions are constant.
  • Multi-pass optimization: some of the variables might be avoided and inlined after the first pass.
  • Various other peephole optimizations: Boolean.valueOf(const), ‘not’ expressions (!const, !!a, !(a==b), !(a!=b), !(a>b), etc.), ‘?’ expressions coming from CASE (a ? booleanConstant : b and a ? b : booleanConstant).
  • Implement left, right and full outer join.
  • Clean build on cygwin/Windows.

0.2 / 2014-04-11

  • Fix #8, “Javadoc generation fails under JDK 1.8”.
  • Fix #15, “Expressions.ifThenElse does not work”.
  • Use HashMap for searching of declarations to reuse; consider both optimizing and optimize flags when reusing.
  • Implement equals and hashCode for expressions. Hash codes for complex expressions are cached into a field of the expression.
  • Add example, com.example.Linq4jExample.
  • Fix optimizing away parameter declarations in assignment target.
  • Support Windows path names in checkstyle-suppresions.
  • Support Statement.toString via ExpressionWriter.
  • Use AtomicInteger for naming of ParameterExpressions to avoid conflicts in multithreaded usage
  • Cleanup: use Functions.adapt rather than new AbstractList
  • Add NOTICE and LICENSE files in generated JAR file.
  • Optimize select() if selector is identity.
  • Enable checkstyle.

0.1.13 / 2014-01-20

  • Remove spurious “null” generated when converting expression to string.
  • Allow a field declaration to not have an initializer.
  • Add Primitive.defaultValue.
  • Enable oraclejdk8 in Travis CI.

0.1.12 / 2013-12-07

  • Add release notes.
  • Fix implementation of Enumerable.asEnumerable in DefaultQueryable (inherited by most classes that implement Queryable).

0.1.11 / 2013-11-06

  • Initial commit