What's New in v2.1.0-alpha.20180730

July 25, 2018

For our July 30th alpha release, in addition to PostgreSQL compatibility enhancements, general usability improvements, and bug fixes, we want to highlight some major benefits:

• Troubleshoot performance problems with hardware metrics - The new Web UI Hardware dashboard provides more visibility into how cluster CPU, networking, disk, and memory resources are being utilized so you can quickly identify and remove performance bottlenecks.
• Easier PostgreSQL migration - We’ve made further enhancements to reduce PostgreSQL migration friction. Notable improvements include support for foreign keys, sequences, and COPY in IMPORT ... PGDUMP.
• Monitor Kubernetes-orchestrated clusters with Prometheus - We expanded our guides for running CockroachDB on Kubernetes in production to include setting up monitoring and alerting with Prometheus and Alertmanager.

Please give these features and the ones below a try. If you see something that can be improved, we’d love to hear from you on GitHub or the Forum.

Get future release notes emailed to you:

Thanks!

Downloads

General changes

• The cost-based optimizer is now on by default. #26893
• The time series system used by CockroachDB to store internal monitoring data now utilizes pre-computed rollups to significantly increase the duration for which monitoring data is available while using less storage. Monitoring data will be available for up to a year by default; however, data older than seven days will be stored at a reduced resolution, and thus will only be able to give details about 30 minute intervals. #27121
• Building CockroachDB from source now requires yarn at version 1.7.0 or above. #27262
• Added support for signing server and client certificates by different CAs. #27636

Enterprise edition changes

• Core dumps are now disabled when encryption is enabled. #27426
• CHANGEFEEDs now use an asynchronous Kafka producer, increasing throughput. #27421

SQL language changes

• CockroachDB now supports custom frame specification for window functions using ROWS (fully-supported) and RANGE modes. For RANGE, <value> PRECEDING and <value> FOLLOWING are not supported. #26666
• The SNAPSHOT isolation level has been removed. Transactions that request to use it are now mapped to SERIALIZABLE. #27040.
• When the cost-based optimizer is enabled, it will also affect prepared queries. #27034
• Upon failing to gather data from other nodes, the SHOW CLUSTER QUERIES and SHOW CLUSTER SESSIONS statements now report the details of the error. #26821
• Improved the description for the age() built-in function. #27082
• The pg_get_indexdef() built-in function now supports 3 arguments. #27161
• Added COPY support to IMPORT .. . PGDUMP. #27062
• The new max_row_size option overrides default limits on line size for IMPORT ... PGDUMP and PGCOPY. #27062
• The SHOW TRACE FOR <stmt> statement was incomplete and incorrect and has thus been removed. To turn on tracing, use SET tracing = ... and SHOW SESSION TRACE, or enable the new auto_trace client-side option for cockroach sql. #26729 #27805
• SET tracing accepts a new option results, which causes result rows and row counts to be copied to the session trace. This was previously implicit with option kv but must now be specified explicitly when desired. #26729
• The word view is now supported as an identifier like in PostgreSQL. #27204
• IMPORT ... PGDUMP no longer requires the --no-owner flag. #27268
• AS OF SYSTEM TIME can now use some more complex expressions to compute the desired timestamp. #27206
• Added support for the convert_from() and convert_to() built-in functions, for compatibility with PostgreSQL. For convert_from(), however, in contrast with PostgreSQL, the function in CockroachDB accepts NUL in the input, because null characters are valid in CockroachDB strings. #27328
• The ALTER ... EXPERIMENTAL CONFIGURE ZONE statement now accepts arbitrary scalar expressions (including possibly containing sub-queries) to compute the YAML operand. #27213
• CockroachDB now recognizes PostgreSQL's abbreviated time units when converting strings to intervals. #27393
• Sorting with a limit and/or input ordering now falls back to disk. #27083
• CockroachDB now reports a hint in the error message if it encounters a correlated query that it does not support yet. #27396
• The new EXPERIMENTAL_RELOCATE LEASE command for ALTER TABLE and ALTER INDEX allows manually transferring the leases for specific ranges to specific stores. #26436
• Added the sql.distsql.flow_stream_timeout and sql.distsql.max_running_flows cluster settings to fine-tune flow setup #27404
• IMPORT now supports a WITH oversample = ... option to decrease variance in data distribution during processing. #27341
• IMPORT ... PGDUMP now supports foreign keys. #27425
• IMPORT ... PGDUMP now supports sequences. #27739
• IMPORT ... PGDUMP now supports empty and public schemas. #27782
• SHOW JOBS now reports results even when a job entry is incomplete or incorrect. #27430
• The column labels in the output of EXPLAIN and all SHOW statements have been renamed for consistency. #27098
• The column labels in the output of SHOW COLUMNS have been renamed for consistency with information_schema. The new generation_expression column reports the expression used for computed columns. #27098
• The SHOW CREATE statement has been simplified and can be used equivalently on tables, views, and sequences without having to specify the type of object to inspect. #27098
• Added the chr() built-in function (the inverse of ascii()). #27278
• Added support for skipping foreign keys in IMPORTs that support them. #27606
• The new sql.optimizer.count metric has been added to track the number of queries with the experimental cost-based optimizer. #26981
• More statement types are now reported in the collected statement statistics in the web UI and diagnostics reporting. #27646
• Added support for KV traces (SHOW KV TRACE FOR SESSION) on DistSQL-executed queries. #27802
• The return type of single-column generator functions has been changed from tuple{columnType} to columnType. This is a compatibility change to match the behavior of PostgreSQL. #27773

Command-line changes

• CockroachDB now computes the correct number of replicas on down nodes. Therefore, when decommissioning nodes via the cockroach node decommission command, the --wait=all option no longer hangs indefinitely when there are down nodes. As a result, the --wait=live option is no longer necessary and has been deprecated. The --wait=all option is now the default. #27027
• Added the cockroach sqlfmt command for formatting SQL statements. #27240
• The output labels of cockroach user ls and cockroach user get have been renamed for consistency with the SQL SHOW USERS statement. Also, to reduce inadvertent data leaks, the output of cockroach user get no longer includes hashed passwords. #27098
• The new client-side option prompt1 can be used to customize the cockroach sql interactive prompt. #27803
• The new auto_trace client-side option can be use to turn on tracing for a cockroach sql session. #27805

Web UI changes

• The new Hardware dashboard displays time series data about CPU, memory, and disk and network IO. #27626
• Time series metric metadata is now available at <Web UI>/_admin/metricmetada. #25359
• Encryption progress is now reported on <Web UI>/#/reports/stores/local debug page. #26802
• Statement statistics can now be filtered by app on the Statements page. #26949
• Improved the readability of the mean and standard deviation bar chart on the Statement Details page. #26949
• Added a visualization of the standard deviation of the latency of statements to the Statements page. #26949
• The Statements page now shows statements that executed on all nodes in the cluster, not just the gateway node. #26605
• The Statement Details page now includes a table showing statistics broken down by which node was the gateway node. #26605

Bug fixes

• Fixed the ordering of columns in the cockroach node status output. #27042
• Fixed a bug that would make the Statement Details page in the Web UI break if a statement wasn't found. #27105
• Fixed some incorrectly typed columns in the pg_index virtual table. #27723
• Fixed permissions and audit logging issues with the optimizer. #27108
• Prevented a situation in which ranges repeatedly fail to perform a split. #26934
• Fixed a crash that could occur when distributed LIMIT queries were run on a cluster with at least one unhealthy node. #26950
• Failed IMPORTs now begin to clean up partially imported data immediately and in a faster manner. #26959
• IMPORT now detects node failure and will restart instead of fail. #26881
• Fixed a panic in the optimizer with IN filters. #27053
• Fixed a panic that could occur when renaming a scalar function used as a data source. #27039
• The server will not finalize a version upgrade automatically and erroneously if there are nodes temporarily inactive in the cluster. #26821
• The DISTINCT ON clause is now reported properly in statement statistics. #27221
• Fixed a panic in IMPORT when creating a table using a sequence operation (e.g., nextval()) in a column's DEFAULT expression. #27122
• SET now properly rejects attempts to use invalid variable names starting with tracing.. #27216
• Fixed NULL equality handling in the experimental lookup join feature. #27336
• ALTER ... EXPERIMENTAL CONFIGURE ZONE is now properly tracked in statement statistics. #27213
• Invalid uses of set-generating functions in FROM clauses are now reported with the same error code as PostgreSQL. #27390
• The number of COPY columns is now correctly verified during IMPORT ... PGDUMP. #27345
• CHANGFEEDs now correctly emit all versions of quickly changing rows. #27612
• Alleviated a scenario in which a large number of uncommitted Raft commands could cause memory pressure at startup time. #27009
• Prevented the unbounded growth of the Raft log caused by a loss of quorum. #27774
• Foreign key references in IMPORT ... PGDUMP are now processed in the correct order. #27782

Performance improvements

• Transactional writes are now pipelined when being replicated and when being written to disk, dramatically reducing the latency of transactions that perform multiple writes. This can be disabled using the new kv.transaction.write_pipelining_enabled cluster setting. #26599
• Reduced CPU utilization in clusters with many ranges, also during periods of lease rebalancing. #26910 #26907
• Reduced the memory size of commonly used Request and Response objects. #27112
• Improved low-level iteration performance. #27299
• Prevented a scenario when dropping a table could cause excessive compaction activity that would degrade performance significantly. #27353
• Limited the scanner from running incessantly on stores with 100s of thousands of replicas. #27441
• Prevented dead nodes in clusters with many ranges from causing unnecessarily high CPU usage. #26911
• Significantly reduce CPU usage when a large number of ranges are deleted from a node. #27520
• Min, max, sum, and avg now take linear time when used for aggregation as window functions for all supported window frame options. #26988
• CHANGEFEEDs no longer hold all data for each poll in memory at once, increasing scalability. #27612

Build changes

• Upgraded protobuf to 3.6.0 #26935

Doc updates

• Added a tutorial on benchmarking CockroachDB with TPC-C. [#3281][#3281]
• Expanded the Production Checklist to cover a detailed explanation of network flags and scenarios and updated production deployment tutorials to encourage the use of --advertise-host on node start. #3352
• Expanded the Kubernetes tutorials to include setting up monitoring and alerting with Prometheus and Alertmanager. #3370
• Updated the rolling upgrade tutorial with explicit systemd commands. #3396
• Updated the OpenSSL certificate tutorial to allow multiple node certificates with the same subject. #3423
• Added an example on editing SQL statements in an external editor from within the built-in SQL shell. #3425

Contributors

This release includes 328 merged PRs by 42 authors. We would like to thank the following contributors from the CockroachDB community, with special thanks to first-time contributors Art Nikpal, Ivan Kozik, Tarek Badr, and nexdrew.

• Art Nikpal
• Brett Snyder
• Ivan Kozik
• Nishant Gupta
• Tarek Badr
• neeral
• nexdrew