MySQL 5.7.21

What's new in this version:

Audit Log Notes:
- MySQL Enterprise Audit now supports compression and encryption of audit log files. Encryption is based on a user-defined password. To use this feature, the MySQL keyring must be enabled because audit logging uses it for password storage. MySQL Enterprise Audit also now supports logging in JSON format, in addition to the existing XML formats. For JSON format, functions are available that provide runtime log reading capabilities. For additional information, see MySQL Enterprise Audit.Note
- Compared to previous MySQL versions, interpretation of the log file name (the audit_log_file system variable value) has changed, as has log file renaming behavior at audit log plugin initialization and termination. See Audit Log File Name

Configuration Notes:
- For RHEL, SLES, and Fedora RPMs, the default plugin directory for debug builds has been changed from /usr/lib64/mysql/plugin to /usr/lib64/mysql/plugin/debug
- The installation scripts for MySQL Enterprise Audit and MySQL Enterprise Firewall now create their associated tables in the mysql system database as InnoDB rather than MyISAM tables
- The hardcoded memory page size of 8KB for the memory-mapped transaction coordinator was too small for platforms such as ARM64 and PowerPC where the page size is much larger. The server now invokes a system call to get the page size of the current platform rather than using a hardcoded value. A consequence for the --log-tc-size option is that the minimum and default values are now 6 times the page size. Also, the value must be a multiple of the page size. Thanks to Alexey Kopytov for the patch

Performance Schema Notes:
- The Performance Schema setup_timers table is now deprecated, to be removed in MySQL 8.0, as is the TICK row in the performance_timers table

Pluggable Authentication:
- For the LDAP authentication plugins, handling of the group search attribute indicated by the authentication_ldap_sasl_group_search_attr and authentication_ldap_simple_group_search_attr system variables is more flexible. If the group search attribute is isMemberOf, LDAP authentication directly retrieves the user attribute isMemberOf value and assign it as group information. If the group search attribute is not isMemberOf, LDAP authentication searches for all groups where the user is a member. (The latter is the default behavior.) This behavior is based on how LDAP group information can be stored two ways: 1) A group entry can have an attribute named memberUid or member with a value that is a user name; 2) A user entry can have an attribute named isMemberOf with values that are group names
- The LDAP authentication plugins now permit the authentication string that provides user DN information to begin with a + character. In the absence of this character, the authentication string value is treated as is without modification, as it has been previously. If the authentication string begins with +, the plugin constructs the full user DN value from the account user name as the cn attribute value, together with the authentication string (with the + removed). The authentication string is stored as given in the mysql.user system table, with the full user DN constructed on the fly before authentication
- For the LDAP authentication plugins, the group search attribute was fixed and not configurable. Two new system variables now enable using custom group filters: authentication_ldap_sasl_group_search_filter and authentication_ldap_simple_group_search_filter. (Bug #26091340)

Security Notes:
- Incompatible Change: Passwords are now restricted to a maximum of 256 characters for the sha256_password authentication plugin, and for the PASSWORD() function when old_passwords=2. Also, the number of password hashing rounds is capped to limit CPU time used
- The linked OpenSSL library for the MySQL Commercial Server has been updated to version 1.0.2n
- This change does not affect the Oracle-produced MySQL Community build of MySQL Server, which uses the yaSSL library instead
- MySQL now supports key migration between underlying keyring keystores. This enables DBAs to switch a MySQL installation from one keyring plugin to another. See Migrating Keys Between Keyring Keystores
- MySQL Enterprise Edition now includes a keyring plugin, keyring_encrypted_file, that is similar to the keyring_file plugin in its use of a local data file for key storage, but that also encrypts the file based on a user-defined password. See Using the keyring_encrypted_file Keyring Plugin

Test Suite Notes:
- Documentation for the MySQL Test Suite is now maintained in the MySQL source tree using Doxygen. The related Unix man pages that previously were produced from the old test suite manual are no longer updated and have gone out of date. Consequently, they are no longer included in MySQL distributions
- The MySQL test suite now includes CRC32() tests. Thanks to Daniel Black for the patch

Functionality Added or Changed:
- InnoDB: The innodb_undo_tablespaces configuration option is deprecated and will be removed in a future release
- Replication: Host names can now be specified as part of a whitelist for group replication connections, using the group_replication_ip_whitelist system variable. Host names support CIDR notation. Host names that resolve to IPv6 addresses are not supported. For host names, name resolution takes place only when a connection request is made by another server. A host name that cannot be resolved is not considered for whitelist validation, and a warning message is written to the error log. Forward-confirmed reverse DNS (FCrDNS) verification is carried out for resolved host names. Warning: Host names are inherently less secure than IP addresses in a whitelist. FCrDNS verification provides a good level of protection, but can be compromised by certain types of attack. Specify host names in your whitelist only when strictly necessary, and ensure that all components used for name resolution, such as DNS servers, are maintained under your control. You can also implement name resolution locally using the hosts file, to avoid the use of external components
- Replication: The group_replication_allow_local_disjoint_gtids_join system variable has been deprecated and is scheduled for removal in a future version
- The -DWITH_ASAN_SCOPE CMake option enables the AddressSanitizer -fsanitize-address-use-after-scope Clang flag for use-after-scope detection. The default is off. To use this option, -DWITH_ASAN must also be enabled

Bugs Fixed:
- Important Change; Partitioning: Checking for tables that used the generic partitioning handler could cause delays of several minutes when starting the MySQL Server. To keep this from happening, the --disable-partition-engine-check option is now enabled by default
- Performance; JSON: Creating a representation of a JSON string now optimizes for the most common case—that the string to be processed contains no special characters that need to be escaped—scanning for the first special character in the string, and copying each sequence of characters which do not require escaping in a single memcpy() call, rather than checking each character in turn to determine whether it needed to be escaped, escaping it if so, and then copying it, one by one, as was done previously
- This fix also corrects a failure to escape the control character u001f, or unit separator character
- InnoDB: Multiple updates from different clients on a partitioned table caused an unexpected lock wait timeout due to an incorrectly set lock type
- InnoDB: An ALTER TABLE operation caused the server to halt
- InnoDB: The innodb_table_stats data dictionary table was not updated with new partition names when renaming a partitioned table
- InnoDB: A FLUSH TABLES operation failed to drop an aborted index. While removing the table from the cache, the clustered index was dropped prior to checking for the aborted index
- InnoDB: An iterative approach to processing foreign cascade operations resulted in excessive memory use
- InnoDB: An INSERT operation on table with spatial index raised an assertion due to a failure that occurred during a lock conflict check
- InnoDB: Warnings that should only appear in a debug version of MySQL were printed to the error log when the length of the history list exceeded 2000000
- InnoDB: Attempting to reduce the buffer pool size to less than the buffer pool chunk size did not report a warning
- InnoDB: A “wrong key column” error was added to address an unsupported index creation scenario
- InnoDB: Full-text search on indexed columns that use a binary collation did not return case sensitive matches
- Packaging: When trying to install MySQL Server on Fedora 27 using the MySQL Yum repository, installation failed due to a conflict with the native mariadb-connector-c-devel package. With this fix, the appropriate “obsoletes” have been added for that and other native packages
- Replication: All servers that belong to a group must have unique UUIDs set by server_uuid, but this was not being enforced by Group Replication and it was possible to add members with duplicated UUIDs
- Replication: During distributed recovery as part of joining the group, when the applier was signaling that it had applied all transactions, it was also blindly searching for partial transactions. This was to avoid future applier errors, which would happen if the applier stopped at this point. However, this search and remove only made sense for applier stop cases. Upon execution completeness it should not be done, otherwise it can corrupt or purge the applier relay log, which can led to data loss. To solve this issue, when the applier is waiting for execution completeness, it no longer searches for and removes partial transactions
- Replication: Group Replication executes internal operations on the server during start and stop of the plugin, such as enabling or disabling read only mode, using an internal session. When this internal session was opened, if the total number of sessions exceeded the number of permitted open sessions set by max_connections, the operation was failing as expected but a thread was left behind, which later would cause issues
- Replication: The fix for Bug #26117735 (MySQL Bug 86288) could cause a debug assertion when running mysqlbinlog with the --read-from-remote-server option and the --rewrite-db option, depending on the database names specified in the rewrite rule. The issue has now been corrected
- Replication: With MySQL compiled using yaSSL, and semisynchronous replication in use, a deadlock could be caused by incorrect handling of acknowledgement packets. Multiple acknowledgement packets can be read together by yaSSL, but the receiver thread for semisynchronous replication only handled the first acknowledgement packet seen after polling. Now, the receiver thread handles all acknowledgement packets that are present in the buffer
- Replication: If Group Replication was configured to start on server boot when the server was being initialized using --initialize or --initialize-insecure, because the replication applier infrastructure was not initialized this resulted in an assertion. Now, Group Replication is not started when the server is being initialized
- Replication: In a group with heavy load, joining members could need to retrieve a large amount of data to gain synchrony with the group. If the amount of data retrieved exceeded the View_change packet size of 4Mb the members would fail to join the group and enter Error state. Now, the packet size is taken from slave_max_allowed_packet, which defaults to 1GB. Depending on the load your group processes, you might want to increase the packet size further by configuring slave_max_allowed_packet
- Replication: With semisynchronous replication in use, if RESET MASTER was issued while an active transaction was waiting for an acknowledgement from the slave, the count of waiting sessions in the Rpl_semi_sync_master_wait_sessions server status variable was incorrect after the wait was completed
- Replication: In a group where a joining member consistently received transactions, the joining member could sometimes not enter the online state. This was due to the way the incoming queue of messages was tested
- Replication: XA ROLLBACK statements that failed because an incorrect transaction ID was given, could be recorded in the binary log with the correct transaction ID, and could therefore be actioned by replication slaves. A check is now made for the error situation before binary logging takes place, and failed XA ROLLBACK statements are not logged
- Replication: The receiver thread for semisynchronous replication was not able to receive acknowledgements from slaves that used compression of the master/slave protocol (slave_compressed_protocol = ON). The receiver thread now handles compressed acknowledgements correctly
- Replication: On replication slaves, in the XA_STATE field in the Performance Schema table events_transactions_current, the state of XA transactions was incorrectly reported as COMMITTED instead of PREPARED after the XA PREPARE statement was applied on the slave
- Replication: In a multi-source replication topology, a memory leak could occur on the slave when binlog_rows_query_log_events was enabled on the master, and a statement already applied from another channel was skipped on the slave. In this situation, the instance of the Rows_query log event stored on the slave was not being deleted. The log event instance is now cleaned up and the memory is freed. Thanks to Vlad Lesin for his contribution to the patch
- Replication: A memory leak was fixed in GTID-based replication. Memory was not being freed after the repository tables were updated for skipped or ignored events
- Replication: When a worker thread on a multi-threaded slave failed to apply a transaction on which a later transaction depended, the coordinator thread could begin scheduling the dependent transaction before being notified of the issue. If a STOP SLAVE request was made during this situation, it caused an assertion to be raised in debug builds
- Replication: When group_replication_enforce_update_everywhere_checks=ON the Group Replication plugin checks if there are foreign key cascades and disallows updates to such tables. However SET NULL operations were not being checked, which could cause data inconsistency. Now, when group_replication_enforce_update_everywhere_checks=ON, operations on child tables are blocked if the table has a SET NULL option configured
- Replication: On Windows, any errors generated by Group Replication now contain the detailed error message rather than just the error number
- Replication: With statement-based replication in use, if an UPDATE or DELETE statement was used inside an XA transaction ending with XA COMMIT ONE PHASE, and the statement did not affect any rows, a replication error occurred. An XA END statement was not written to the binary log, so slave servers identified the XA transaction as still being active at the time of the commit request. The required XA END statement is now written even if no rows were affected by the transaction
- Replication: Regardless of the number of virtual IPs configured on a machine, Group Replication could access only the first 12 addresses
- Microsoft Windows: On Windows, with the myisam_use_mmap and flush system variables enabled, MyISAM did not always flush table files properly
- Microsoft Windows: On Windows, resolution was improved of a timer used for query performance assessment
- JSON: When inserting JSON values created from the result of a GROUP BY query, the inserted values could sometimes include the concatenation of all the values previously inserted into that column
- In event items in filter rules, the audit_log plugin did not properly process values specified as a JSON array
- VALUES() was not handled correctly in some cases
- In some cases, virtual generated column expressions containing comparison operators could cause problems with subsequent statements accessing the same table
- For debug builds, validation checks on relevant generated columns could be missed for UPDATE statements, leading to a server exit
- The default value of the authentication_ldap_sasl_auth_method_name system variable was incorrectly set to SIMPLE rather than SCRAM-SHA-1, and the variable could be set to impermissible values
- Following an INSERT statement with BLOB values in the ON DUPLICATE KEY UPDATE clause that failed with a constraint violation, a similar statement with no reason to return an error could cause a server exit
- The Performance Schema now stores rewritten rather than raw SQL statement text when available
- Incorrect results or a server exit could result when SHA2() was passed a user-defined variable in some character sets
- mysqlpump no longer includes the SQL_NO_CACHE modifier in statements because that modifier is now deprecated and results in deprecation warnings
- Building with the -DWITHOUT_SERVER=ON CMake option failed due to attempting to link the authentication_ldap_sasl_client client-side plugin against the embedded server library
- Setting authentication_ldap_simple_max_pool_size=0 and authentication_ldap_simple_init_pool_size=0 at runtime did not disable the LDAP connection pool for the authentication_ldap_simple authentication plugin
- Accounts that use an LDAP authentication plugin and were created without any authentication string could be authenticated by the LDAP server regardless of password specified at connect time
- Incorrect results could be returned for queries that used an outer join and a derived table referenced a const value from an inner table of the outer join
- AFTER UPDATE triggers were not invoked for INSERT ... ON DUPLICATE KEY UPDATE when the value to be updated and the new value were the same
- Changing the UMASK and UMASK_DIR environment variables from their default values had no effect on database directory and table file access
- Creating a table with excessive index information could cause a server exit
- MSI packages for Windows failed to detect when Microsoft Visual C++ 2010 Redistributable Package was installed
- audit_log plugin THD objects could be created with incorrect thread ID information, leading to assertion failure
- When HASH_SCAN was specified as one of the values for the slave_rows_search_algorithms system variable, which is the default from MySQL 8.0.2, and row-based replication was in effect, updates to a table containing virtual generated fields could raise an assertion. The issue was caused by an error when generating string representations of the virtual generated fields in order to create hashes for use in searches. To remove the issue, MySQL no longer creates hashes for virtual generated fields
- Attempting a partial backup with mysqlpump on a GTID-enabled server failed and produced an error message suggesting incorrectly that this was not possible. (It is possible using the --set-gtid-purged option.)
- The Performance Schema could leak memory due to nondeletion of file instances created for ALTER TABLE operations that used the table-copy algorithm
- mysqlpump did not properly parse TABLESPACE clauses in the result from SHOW CREATE TABLE statements it executed to determine table structure
- Some statements could cause a buffer overflow in the digest code. Thanks to Laurynas Biveinis and Roel van de Paar for the patch
- Previously, when the Performance Schema failed to initialize, it wrote a nonspecific init failed warning to the error log. Now it prints more specific messages about which memory allocation failed
- Incorrect results could occur on a table with a unique index when the optimizer chose a loose index scan even though the unique index had no index extensions
- Restarting the LDAP server could cause LDAP authentication plugins that used a connection pool to fail to authenticate properly
- CREATE USER IF NOT EXISTS was not written to the binary log if the user existed. This could result in inconsistent replication behavior if the user did not exist on slave servers. A similar issue occurred for ALTER USER IF EXISTS. To avoid inconsistencies, these statements now are written to the binary log
- Incorrect handling of internal memory buffers could cause a server exit
- MySQL did not compile with GCC 7
- Executing a stored procedure containing a statement that created a table from the contents of certain SELECT statements could result in a memory leak
- When an UPDATE required a temporary table having a primary key larger than 1024 bytes and that table was created using InnoDB, the server could exit
- For geometry calculations, invalid input parameters could lead to an incorrect result buffer and cause an assertion to be raised or a server exit
- Under some conditions, the audit_log plugin could recursively lock a mutex, resulting in an unresponsive server
- In some cases, the optimizer chose a loose index scan (QUICK_GROUP_MIN_MAX_SELECT) for a GROUP BY query even when there was a predicate with a disjunction. This is fixed by not performing a range scan when the condition in the WHERE clause results in more than one disjoint range tree
- Setting the MYSQL_GROUP_SUFFIX environment variable had no effect
- Queries with many left joins were slow if join buffering was used (for example, using the block nested loop algorithm)
- A prepared statement containing an ORDER BY list that referred to a parameter was not always handled correctly
- The server handled triggers and generated columns incorrectly