What's New in v2.0-beta.20180305

March 5, 2018

This week's release includes:

• Improved support for large delete statements.
• Reduced disruption during upgrades and restarts.
• Reduced occurrence of serializable transaction restarts.

Get future release notes emailed to you:

Thanks!

Downloads

Backwards-Incompatible Changes

• Sequences are now supported in enterprise BACKUP/RESTORE jobs.

  This changes how sequences are stored in the key-value storage layer, so existing sequences must be dropped and recreated. Since a sequence cannot be dropped while it is being used in a column's DEFAULT expression, those expressions must be dropped before the sequence is dropped, and recreated after the sequence is recreated. The setval() function can be used to set the value of a sequence to what it was previously. #21684

• Positive constraints in replication zone configs no longer work. Any existing positive constraints will be ignored. This change should not impact existing deployments since positive constraints have not been documented or supported for some time. #22906

Build Changes

• CockroachDB now builds with go 1.9.4 and higher. #22608

General Changes

• Diagnostics reports now include information about changed settings and statistics on types of errors encountered during SQL execution. #22705, #22693, #22948

Enterprise Edition Changes

• Revision history BACKUP/RESTORE is no longer considered experimental. #22679
• Revision history BACKUP/RESTORE now handles schema changes. #21717
• CockroachDB now checks that a backup actually contains the requested restore time. #22659
• Improved BACKUP's handling of tables after TRUNCATE. #21895
• Ensured that only the backups created by the same cluster can be used in incremental backups. #22474
• Avoided extra internal copying of files during RESTORE. #22281
• Added a geographical map to the homepage of the Admin UI enterprise version, showing the location of nodes and localities in the cluster. The map is annotated with several top-level metrics: storage capacity used, queries per second, and current CPU usage, as well as the liveness status of nodes in the cluster. [#22763][#22763]

SQL Language Changes

• The type determined for constant NULL expressions is renamed to unknown for better compatibility with PostgreSQL. #23150
• Deleting multiple rows at once now consumes less memory. #23013
• Attempts to modify virtual schemas with DDL statements now fail with a clearer error message. #23041
• The new SHOW SCHEMAS statement reveals which are the valid virtual schemas next to the physical schema public. #23041
• CockroachDB now recognizes the special syntax SET SCHEMA <name> as an alias for SET search_path = <name> for better compatibility with PostgreSQL. #23041
• current_role() and current_catalog() are supported as aliases for the current_user() and current_database() built-in functions for better compatibility with PostgreSQL. #23041
• CockroachDB now returns the correct error code for division by zero. #22948
• The GC of table data after a DROP TABLE statement now respects changes to the GC TTL interval specified in the zone config #22903
• The full names of tables/view/sequences are now properly logged in the system event log. #22848
• CockroachDB now recognizes the syntax db.public.tbl in addition to db.tbl for better compatibility with PostgreSQL. The handling of the session variable search_path, as well as that of the built-in functions current_schemas() and current_schema(), is now closer to that of PostgreSQL. Thus SHOW TABLES FROM can now inspect the tables of a specific schema (for example, SHOW TABLES FROM db.public or SHOW TABLES FROM db.pg_catalog). SHOW GRANTS also shows the schema of the databases and tables. #22753
• Users can now configure auditing per table and per access mode with ALTER TABLE. #22534
• SQL execution logs enabled by the cluster setting sql.trace.log_statement_execute now go to a separate log file. This is an experimental feature meant to aid troubleshooting CockroachDB. #22534
• Added the string_to_array() built-in function. #22391
• Added the constraint_column_usage table and roles-related tables to the information_schema database. #22323 #22242
• IMPORT no longer requires the experimental setting. #22531
• Computed columns and CHECK constraints now correctly report column names in the case of a type error. #22500
• The output of JSON data now matches that of PostgreSQL. #22393
• Allowed IMPORT jobs to be paused. IMPORT jobs now correctly resume instead of being abandoned if the coordinator goes down. #22291
• Removed the into_db option in IMPORT. The database is now specified as part of the table name. #21813
• Changed computed column syntax and improved related error messages. #22429
• Implemented additional INET column type operators such as contains and contained by, binary operations, and addition/subtraction.
• Implemented the following operators for INET column types: <<, <<=, >>, >>=, &&, +, -, ^, |, &. These operators are compatible with PostgreSQL 10 and are described in Table: 9.36 of the PostgreSQL documentation. #21437
• CockroachDB now properly rejects incorrectly-cased SQL function names with an error. #22365
• Allowed DEFAULT expressions in the CREATE TABLE of an IMPORT CSV. The expressions are not evaluated (data in the CSV is still required to be present). This change only allows them to be part of the table definition. #22307
• Added the #- operator for JSON. #22375
• The SET transaction_isolation statement is now supported for better PostgreSQL compatibility. #22389
• Allowed creation of computed columns. #21823
• Avoided extra internal copying of files during IMPORT. #22281
• Casts between array types are now allowed when a cast between the parameter types is allowed. #22338
• Scalar functions can now be used in FROM clauses. #22314
• Added privilege checks on sequences. #22284
• The ON DELETE SET DEFAULT, ON UPDATE SET DEFAULT, ON DELETE SET NULL, and ON UPDATE SET NULL foreign key constraint actions are now fully supported. #22220 #21767 #21716
• The ON DELETE CASCADE and ON UPDATE CASCADE foreign key constraint actions can now also contain CHECK constraints. #22535
• JSON inverted indexes can now be specified in a CREATE TABLE statement. #22217
• When a node is gracefully shut down, planning queries are avoided and distributed queries are allowed the amount of time specified by the new server.drain_max_wait cluster setting before the node is drained and stopped. #20450
• Collated string are now supported in IMPORT jobs. #21859
• The new SHOW GRANTS ON ROLE statement and pg_catalog.pg_auth_members table lists role memberships. #22205 #21780
• Role memberships are now considered in permission checks. #21820

Command-Line Changes

• Replication zone constraints can now be specified on a per-replica basis, meaning you can configure some replicas in a zone's ranges to follow one set of constraints and other replicas to follow other constraints. #22906
• Per-replica constraints no longer have to add up to the total number of replicas in a range. If you do not specify all the replicas, then the remaining replicas will be allowed on any store. #23081
• cockroach sql now reminds the user about SET database = ... and CREATE DATABASE if started with no current database set. #23089
• Error messages displayed while connecting to a server with an incompatible version have been improved. #22709
• The --cache and --max-sql-memory flags of cockroach start now also support decimal notation to support a fraction of total available RAM size, e.g., --cache=.25 is equivalent to --cache=25%. This simplifies integration with system management tools. #22460
• When printing tabular results as CSV or TSV, no final row count is emitted. This is intended to increase interoperability with external tools. #20835
• The pretty formatter does not introduce special unicode characters in multi-line table cells, for better compatibility with certain clients. To disambiguate multi-line cells from multiple single-line cells, a user can use WITH ORDINALITY to add a row numbering column. #22324
• Allowed specification of arbitrary RocksDB options. #22401
• Debug commands now open RocksDB in read-only mode. This makes them faster and able to run in parallel. #21778
• The cockroach dump command now outputs CREATE SEQUENCE statements before the CREATE TABLE statements that use them. #21774
• For better compatibility with psql's extended format, the table formatter records now properly indicates line continuations in multi-line rows. #22325
• The cockroach sql client-side option show_times is now always enabled when output goes to a terminal, not just when display_format is set to pretty. #22326
• When formatting cockroach sql results with --format=sql, the row count is now printed in a SQL comment at the end. #22327
• When formatting cockroach sql results with --format=csv or --format=tsv, result rows that contain special characters are now quoted properly. #19306

Admin UI Changes

• Decommissioned nodes are no longer included in cluster stats aggregates. #22711
• Time series metrics dashboards now show their own title rather than the generic "Cluster Overview". #22746
• The URLs for Admin UI pages have been reorganized to provide more consistent structure to the site. Old links will redirect to the new location of the page. #22746
• Nodes being decommissioned are now included in the total nodes count until they are completely decommissioned. #22690
• Added new graphs for monitoring activity of the time series system. #22672
• Disk usage for time series data is now visible on the Databases page. #22398
• Added a ui-clean task. #22552
• Added an icon to indicate when descriptions in the Jobs table are shortened and expandable. #22221
• Added "compaction queue" graphs to the Queues dashboard. #22218
• Added Raft snapshot queue metrics to the Queue dashboard. #22210
• Dead nodes are now displayed before live nodes on the Nodes List page. #22222
• Links to documentation in the Admin UI now point to the docs for v2.0. #21894

Bug Fixes

• Fixed an issue where Admin UI graph tooltips would continue to display zero values for nodes which had long been decommissioned. #22626
• Fixed an issue where Admin UI graphs would occasionally have a persistent "dip" at the leading edge of data. #22570
• Fixed an issue where viewing Admin UI graphs for very long time spans (e.g., 1 month) could cause excessive memory usage. #22392
• Fixed padding on the Node Diagnostics page of the Admin UI. #23019
• Corrected the title of the decommissioned node list, which was mistakenly updated to say "Decommissioning". #22703
• Fixed a bug in cockroach dump output with SEQUENCES. #22619
• Fixed a bug that created uneven distribution of data (or failures in some cases) during IMPORT of tables without an explicit primary key. #22542
• Fixed a bug that could prevent disk space from being reclaimed. #23153
• Replication zone configs no longer accept negative numbers as input. #23081
• Fixed the occasional selection of sub-optimal rebalance targets. #23081
• cockroach dump is now able to dump sequences with non-default parameters. #23062
• Arrays now support the IS [NOT] DISTINCT FROM operators. #23090
• SHOW TABLES is now again able to inspect virtual schemas. #23041
• The special form of CREATE TABLE .. AS now properly supports placeholders in the subquery. #23046
• Fixed a bug where ranges could get stuck in an infinite "removal pending" state and would refuse to accept new writes. #23024
• Fixed incorrect index constraints on primary key columns on unique indexes. #23003
• Fixed a panic when upgrading quickly from v1.0.x to v2.0.x #22971
• Fixed a bug that prevented joins on interleaved tables with certain layouts from working. #22935
• The service latency tracked for SQL statement now includes the wait time of the execute message in the input queue. #22881
• The conversion from INTERVAL to FLOAT now properly returns the number of seconds in the interval. #22894
• Fixed incorrect query results when the WHERE condition contains IN expressions where the right-hand side tuple contains NULLs. #22735
• Fixed incorrect handling for IS (NOT) DISTINCT FROM when either side is a tuple that contains NULL. #22718
• Fixed incorrect evaluation of IN expressions where the left-hand side is a tuple, and some of the tuples on either side contain NULL. #22718
• Expressions stored in check constraints and computed columns are now stored de-qualified so that they no longer refer to a specific database or table. #22667
• Fixed a bug where reusing addresses of decommissioned nodes could cause issues with Admin UI graphs. #22614
• IMPORT jobs can no longer be started if the target table already exists. #22627
• Computed columns can no longer be added to a table after table creation. #22653
• Allowed UPSERTing into a table with computed columns. #22517
• Computed columns are now correctly disallowed from being foreign key references. #22511
• Various primitives that expect table names as argument now properly reject invalid table names. #22577
• AddSSTable no longer accidentally destroys files in the log on success. #22551
• IsDistinctFrom with NULL placeholder no longer returns incorrect results. #22433
• Fixed a bug that caused incorrect results for joins where columns that are constrained to be equal have different types. #22549
• Implemented additional safeguards against RPC connections between nodes that belong to different clusters. #22518
• The /health endpoint now returns a node as unhealthy when draining or decommissioning. #22502
• Aggregates that take null arguments no return the correct results. #22507
• Fixed empty plan columns of sequenceSelectNode. #22495
• Disallowed any inserts into computed columns. #22470
• Tables with computed columns will produce a meaningful dump. #22402
• cockroach sql no longer produces an error anymore when an empty statement is entered at the interactive prompt. #22449
• The pg_typeof() function now returns the correct type for the output of UNION ALL even when the left sub-select has a NULL column. #22438
• <TYPE> <value> literal casts now work correctly for all fixed-length types. #22397
• Errors from DDL statements sent by a client as part of a transaction, but in a different query string than the final commit, are no longer silently swallowed. #21829
• Fixed a bug in cascading foreign key actions. #21799
• Tabular results where the column labels contain newline characters are now rendered properly. #19306
• Fixed a bug that prevented long descriptions in the Admin UI Jobs table from being collapsed after being expanding. #22221
• Fixed a bug that prevented using SHOW GRANTS with a grantee but no targets. #21864
• Fixed a panic with certain queries involving the REGCLASS type. #22310
• Fixed the behavior and types of the encode() and decode() functions. #22230
• Fixed a bug that prevented passing the same tuple for FROM and TO in ALTER TABLE ... SCATTER. #21830
• Fixed a regression that caused certain queries using LIKE or SIMILAR TO with an indexed column to be slow. #21842
• Fixed a stack overflow in the code for shutting down a server when out of disk space #21768
• Fixed Windows release builds. #21793
• Fixed an issue with the wire-formatting of BYTES arrays. #21712
• Fixed a bug that could lead to a node crashing and needing to be reinitialized. #21771
• When a database is created, dropped, or renamed, the SQL session is blocked until the effects of the operation are visible to future queries in that session. #21900
• Fixed a bug where healthy nodes could appear as "Suspect" in the Admin UI if the web browser's local clock was skewed. #22237
• Fixed bugs when running DistSQL queries across mixed-version (1.1.x and 2.0-alpha) clusters. #22897

Performance Improvements

• Improved a cluster's ability to continue operating when nearly out of disk space on most nodes. #21866
• Disk space is more aggressively freed up when the disk is almost full. #22235
• Experimentally enabled some joins to perform a lookup join and increase join speed for cases where the right side of the join is much larger than the left. #22674
• Supported distributed execution of INTERSECT and EXCEPT queries. #22442
• Reduced cancellation time of DistSQL aggregation queries. #22684
• Unnecessary value checksums are no longer computed, speeding up database writes. #22487
• Reduced unnecessary logging in the storage layer. #22516
• Improved the performance of distributed sql queries. #22471
• Distributed execution of INTERSECT ALL and EXCEPT ALL queries is now supported. #21896
• Allowed - in usernames, but not as the first character. #22728
• A COMMIT reporting an error generated by a previous parallel statement (i.e., RETURNING NOTHING) no longer leaves the connection in an aborted transaction state. Instead, the transaction is considered completed and a ROLLBACK is not necessary. #22683
• Significantly reduced the likelihood of serializable restarts seen by clients due to concurrent workloads. #21140
• Reduced disruption from nodes recovering from network partitions. #22316
• Improved the performance of scans by copying less data in memory. #22309
• Slightly improved the performance of low-level scan operations. #22244
• When a range grows too large, writes are now be backpressured until the range is successfully able to split. This prevents unbounded range growth and improves a clusters ability to stay healthy under hotspot workloads. #21777
• The information_schema and pg_catalog databases are now faster to query. #21609
• Reduced the write amplification of Raft replication. #20647

Doc Updates

• Added cloud-specific hardware recommendations. #2312
• Added a detailed listing of SQL standard features with CockroachDB's level of support. #2442
• Added docs on the INET data type. #2439
• Added docs on the SHOW CREATE SEQUENCE statement. #2406
• Clarified that password creation is only supported in secure clusters. #2567
• Added docs on production monitoring tools and the critical events and metrics to alert on. #2564
• Added docs on the JSONB data type. #2561
• Added docs on the BETWEEN SYMMETRIC operator. #2551
• Updated docs on supporting castings for ARRAY values. #2549
• Various improvements to docs on the built-in SQL client. #2544

Contributors

This release includes 430 merged PRs by 37 authors. We would like to thank all contributors from the CockroachDB community, with special thanks to first-time contributors noonan, Mark Wistrom, pocockn, and 何羿宏.