MongoDB 8.0.0

What's new in this version:

Logging:
- Starting in MongoDB 8.0, you can configure the Database Profiler to log slow operations based on the time that MongoDB spends working on that operation, rather than the total latency for the operation. This means that factors such as waiting for locks and flow control do not affect whether an operation exceeds the slow operation threshold.

This change provides the following improvements for logging and query analysis:
- Slow queries are logged more accurately based on the time MongoDB spends processing the query.
- Query analysis tools such as the Query Profiler, Performance Advisor, and Search Query Telemetry report slow operations based on workingMillis instead of durationMillis. This change provides a more accurate view of problematic queries.
- Slow query logs include a metric for the time queued on execution tickets, queues.execution.totalTimeQueuedMicros. This metric helps identify whether an operation is slow because of the time it takes to complete, or the time it spends waiting to start.

Database Profiler:
- When you specify a filter for the database profiler, you can log operations based on the new workingMillis metric. You can log operations based on both workingMillis and durationMillis and set each metric to a different threshold.

Aggregation:
BinData Conversion:
Starting in MongoDB 8.0, you can use the $convert operator to perform the following conversions:
- String values to binData values
- binData values to string values
- MongoDB 8.0 also includes a new helper expression, $toUUID, which provides simplified syntax for converting strings to UUID values.

$queryStats Change Stream Improvements:
- Starting in MongoDB 8.0, $queryStats improves tracking and reporting metrics for change streams. For more information, see $queryStats Change Streams Behavior.

$rank and $denseRank Behavior:
- Starting in MongoDB 8.0, null and missing field values in $denseRank and $rank sortBy operations are treated the same when calculating rankings. This change makes the behavior of denseRank and rank consistent with $sort.

Sharding:
directShardOperations Role:
- Starting in MongoDB 8.0, you can use the directShardOperations role to perform maintenance operations that require you to execute commands directly against a shard.

Unshard Collections:
- Starting in MongoDB 8.0, you can unshard existing sharded collections with the unshardCollection command or the sh.unshardCollection() method. This operation moves all documents in the collection onto a specified shard or the shard with the least amount of data.

Replication:
New replSetGetStatus Metrics:
Starting in MongoDB 8.0, the following metrics are available when using the replSetGetStatus command:
- replSetGetStatus.electionCandidateMetrics.lastSeenWrittenOpTimeAtElection
- replSetGetStatus.electionParticipantMetrics.lastWrittenOpTimeAtElection
- replSetGetStatus.electionParticipantMetrics.maxWrittenOpTimeInSet
- replSetGetStatus.members[n].optimeWritten
- replSetGetStatus.members[n].optimeWrittenDate
- replSetGetStatus.members[n].lastWrittenWallTime
- replSetGetStatus.optimeWritten
- replSetGetStatus.optimeWrittenDate
- replSetGetStatus.optimes.lastWrittenWallTime
- replSetGetStatus.optimes.writtenOpTime

Oplog Buffers:
- Starting in MongoDB 8.0, secondaries write and apply oplog entries for each batch in parallel. A writer thread reads new entries from the primary and writes them to the local oplog. An applier thread asynchronously applies these changes to the local database. This changes increases replication throughput for secondaries.
- This introduces a breaking change for metrics.repl.buffer, as it now provides metrics on two buffers instead of one.

MongoDB 8.0 deprecates the following server status metrics:
- metrics.repl.buffer.count
- metrics.repl.buffer.maxSizeBytes
- metrics.repl.buffer.sizeBytes

MongoDB 8.0 adds the following server status metrics:
- metrics.repl.buffer.apply
- metrics.repl.buffer.apply.count
- metrics.repl.buffer.apply.maxCount
- metrics.repl.buffer.apply.maxSizeBytes
- metrics.repl.buffer.apply.sizeBytes
- metrics.repl.buffer.write
- metrics.repl.buffer.write.count
- metrics.repl.buffer.write.maxSizeBytes
- metrics.repl.buffer.write.sizeBytes
- metrics.repl.write
- metrics.repl.write.batchSize
- metrics.repl.write.batches
- metrics.repl.write.batches.num
- metrics.repl.write.batches.totalMillis

General Changes:
Moving Collections that have Change Streams:
- Starting in MongoDB 8.0, movePrimary doesn't invalidate collections that have change streams. The change streams can continue to read events from collections after the collections are moved to a new shard.

New Database Commands:
- abortMoveCollection
- abortUnshardCollection
- moveCollection
- unshardCollection

New mongosh Methods:
- sh.abortMoveCollection()
- sh.abortUnshardCollection()
- sh.moveCollection()
- sh.shardAndDistributeCollection()
- sh.unshardCollection()

Moveable Collections:
- Starting in MongoDB 8.0, you can move an unsharded collection to a different shard using the moveCollection command.

Shutdown Performance:
- Starting in MongoDB 8.0, Bulk.insert() and data ingestion workloads may perform better. However, shutting down MongoDB immediately after running these workloads might take longer because of the extra data being flushed to disk.

Store Application Data on Config Shards:
- Starting in MongoDB 8.0, you can configure a config server to store application data in addition to the usual sharded cluster metadata. The config server is then known as a config shard. For details, see Config Shards.

$lookup Stage in Transactions with Sharded Collections:
- Starting in MongoDB 8.0, you can use the $lookup stage within a transaction while targeting a sharded collection.

Compaction Improvements:
Background Compaction:
- Starting in MongoDB 8.0, you can use the new autoCompact command to perform background compaction. If enabled, the server attempts to keep free space within each collection and index below the specified the freeSpaceTargetMB value.

dryRun Option:
- If enabled, the compact command returns an estimate of how much space, in bytes, compaction can reclaim from the targeted collection. If you run compact with dryRun set to true, MongoDB only returns the estimated value and does not perform any kind of compaction.

Parameter Filtering:

Starting in MongoDB 8.0, the getParameter command accepts a setAt field. You can use this field to filter the allParameters: true return document to show only those parameters that you can set at startup or runtime.

New Bulk Write Command:
- Starting in MongoDB 8.0, you can use the new bulkWrite command to perform many insert, update, and delete operations on multiple collections in one request. The existing db.collection.bulkWrite() method only allows you to modify one collection in one request.

Read Concern on Capped Collections:
- Starting in MongoDB 8.0, you can use read concern "snapshot" on capped collections.

DDL Operations:
- In MongoDB 8.0, if you add or remove a shard while your cluster executes a DDL operation (operation that modifies a collection such as reshardCollection), any operation that adds or removes a shard only executes after the concurrent DDL operation finishes.

Default Timeout for Queries:
- Starting in MongoDB 8.0, you can use the defaultMaxTimeMS cluster parameter to specify a default time limit for individual read operations to complete.

New Query Shape and Query Settings:
- MongoDB 8.0 introduces a new query shape. The pre-existing query shape is renamed as the plan cache query shape.
- Starting in MongoDB 8.0, you can add query settings for the new query shape. The query optimizer uses the query settings as an additional input during query planning, which affects the plan selected to run a query that has a matching query shape.
- Query settings have improved functionality compared to index filters. Index filters are also deprecated starting in MongoDB 8.0, and you should avoid using them.
- To add query settings, use the setQuerySettings command.
- To delete query settings, use the removeQuerySettings command.
- To retrieve query settings, use a $querySettings stage in an aggregation pipeline.
- To block a query shape, see operation rejection filters.

Starting in MongoDB 8.0, queryShapeHash is included in the following output:
- explain command in the explain.queryShapeHash field
- database profiler in the system.profile.queryShapeHash field
- $currentOp aggregation pipeline stage in the $currentOp.queryShapeHash field
- slow query log
- Starting in MongoDB 8.0, the pre-existing queryHash field is renamed to planCacheShapeHash. If you're using an earlier MongoDB version, you'll see queryHash instead of planCacheShapeHash.

Sort Support for updateOne:
- Starting in MongoDB 8.0, the updateOne() method supports a sort option. This controls how MongoDB orders the document before it selects the document to receive the update.
- Previous releases use the findAndModify() and findOneAndUpdate() methods to update the first document in a user-specified sort order. Support for retryable writes requires these methods to copy the entire document to a special side collection for each node, which is a more expensive operation than the updateOne() method with the new sort option.

Upgraded TCMalloc:
- Starting in MongoDB 8.0, MongoDB uses an upgraded version of TCMalloc that uses per-CPU caches, instead of per-thread caches, to reduce memory fragmentation and make your database more resilient to high-stress workloads.

serverStatus Metrics:
The following new serverStatus metrics report information about tcmalloc usage:
- tcmalloc.usingPerCPUCaches
- tcmalloc.generic.peak_memory_usage
- tcmalloc.tcmalloc.cpu_free
- tcmalloc.tcmalloc.total_bytes_held
- tcmalloc.tcmalloc.cpuCache
- tcmalloc.tcmalloc.cpuCache.N.overflows
- tcmalloc.tcmalloc.cpuCache.N.underflows
- tcmalloc.tcmalloc_derived.total_free_bytes

tcmallocEnableBackgroundThread Parameter:
- Starting in MongoDB 8.0, the tcmallocEnableBackgroundThread is enabled by default. This allows MongoDB to periodically release memory back to the operating system.

Query Planning and Execution:
Block Processing:
- Starting in version 8.0, MongoDB may execute certain time series queries using block processing. This performance improvement processes queries in "blocks" of data, rather than individual values. Block processing improves query execution speed and throughput when working with time series collections.
- To see if your time series query uses block processing, check the explain.queryPlanner.winningPlan.slotBasedPlan.stages field in the explain plan output.

Express Query Stages:
Starting in MongoDB 8.0, a limited set of queries (including _id equality matches) skip regular query planning and execution. Instead, these queries use an optimized index scan plan consisting of one of these plan stages:
- EXPRESS_CLUSTERED_IXSCAN
- EXPRESS_DELETE
- EXPRESS_IXSCAN
- EXPRESS_UPDATE

Rejected Query Plan Output:
- Starting in MongoDB 8.0, rejected query plans only contain the find portion of the query. In previous versions, rejected plans can contain aggregation stages like $group. Those aggregation stages aren't used by the query planner to choose the winning plan, so the rejectedPlans field only contains the portion of the query that was used to choose the winning plan.
- This change also ensures that executionStats for rejected plans are non-zero. As a result, you can now see statistics such as how many documents or keys a rejected plan examined.

Batch Multi-Document Insert Operations:
- Starting in MongoDB 8.0, insert operations for multi-document transactions no longer produce individual oplog entries. Instead, multi-document inserts are batched as a single entry. The change stream insert event for each document has the same clusterTime.
- This change increases performance of multi-document insert operations and eliminates possible replication lag caused by multiple oplog writes.

OIDC Identity Providers Can Share an Issuer:
- Starting in MongoDB 8.0, when multiple identity providers (IDP) are defined, the oidcIdentityProviders parameter accepts duplicate issuer values as long as the audience value is unique for each issuer. This is also available in versions 7.3 and 7.0.

Namespaces in Subpipelines:
- Starting in MongoDB 8.0, namespaces in subpipelines inside $lookup and $unionWith are validated to ensure the correct use of from and coll fields.

Security:
Queryable Encryption Range Queries:
- Starting in MongoDB 8.0, Queryable Encryption supports range queries on encrypted fields using the $lt, $lte, $gt, and $gte operators. For details, see Supported Operations for Queryable Encryption. To enable range queries on encrypted fields, see Create an Encryption Schema.

OCSF Schema for Log Messages:
- Starting in MongoDB 8.0, you can specify the OCSF schema for audit log messages. The OCSF schema provides logs in a standardized format compatible with log processors.
- To set the schema used for log messages, use the auditLog.schema configuration file option