fix(firestore): fixes for Query to Pipeline Conversion (#14422) This PR updates the Firestore `toPipeline` logic to ensure that queries converted into pipelines follow standard Firestore query semantics. Previously, the pipeline translation was missing several critical behaviors regarding filtering, sorting, and missing-field handling. All the below gaps were identified by running the Java tests in [ITQueryToPipelineTest.java](https://github.com/googleapis/java-firestore/blob/9065134fd70da22250f038d7e964f26aebfeb3cf/google-cloud-firestore/src/test/java/com/google/cloud/firestore/it/ITQueryToPipelineTest.java) and comparing the execute request payload sent by Java with the ones sent by Go equivalent pipelines. Here are the key improvements: ### 1. Correct Stage Ordering (Filters Before Sorts) We now apply the `Where` filters before the `Sort` stage. Applying sorting before filtering is inefficient and diverges from the standard Firestore execution plan, which filters the dataset as early as possible. **Example:** ```go // Before: collection -> sort("name") -> where("age" == 25) // After: collection -> where("age" == 25) -> sort("name") query := client.Collection("users").Where("age", "==", 25).OrderBy("name", firestore.Asc) pipeline := query.toPipeline() ``` ### 2. Enforcing "Missing Field" Semantics In Firestore, a query for `foo == 1` or an ordering on `foo` should implicitly exclude any document where the field `foo` is missing. We now automatically inject `FieldExists` checks into the pipeline to mirror this behavior. **Example:** * **Filter:** `Where("foo", "==", 1)` $\rightarrow$ `And(FieldExists("foo"), Equal("foo", 1))` * **Order:** `OrderBy("bar", Asc)` $\rightarrow$ `Where(FieldExists("bar")) -> Sort("bar", Asc)` ### 3. Support for Composite Filters (OR / Nested AND) The previous implementation could only handle simple top-level field filters. We've added a recursive filter translator that allows the `toPipeline` method to handle complex nested filters, including `OR` queries. **Example:** ```go // This now correctly translates to a nested 'or' function in the pipeline 'where' stage filter := firestore.OrFilter{ firestore.PropertyFilter{Path: "status", Op: "==", Value: "urgent"}, firestore.PropertyFilter{Path: "priority", Op: ">", Value: 10}, } query := client.Collection("tasks").WhereEntity(filter) ``` ### 4. Native `LimitToLast` Implementation Since the pipeline backend doesn't have a specific "limit to last" stage, we now implement this by: 1. Reversing the sorting order. 2. Applying a standard `Limit`. 3. Re-sorting back to the original requested order. **Example:** ```go // Fetch the 5 most recent items // Pipeline: Sort(timestamp DESC) -> Limit(5) -> Sort(timestamp ASC) query := client.Collection("logs").OrderBy("timestamp", firestore.Asc).LimitToLast(5) ``` ### 5. Correct Path Resolution for Subcollections We fixed an issue where subcollection queries were passing the wrong path to the `collection` stage. It now correctly resolves the relative path from the database root. **Example:** ```go // Previously, this might only pass "comments" to the pipeline. // Now it correctly passes "posts/post_123/comments". query := client.Collection("posts").Doc("post_123").Collection("comments") ``` ### 6. Accurate Cursor Boundaries (StartAt / EndAt) We've refined how multi-field cursors are translated into pipeline inequalities. - **Reference Types:** When using a document as a cursor, we now correctly pass it as a `Reference` type instead of a string ID, ensuring the backend can perform accurate comparisons. - **Strict Inequalities:** For multi-field sorts (e.g., `OrderBy("A").OrderBy("B")`), we now use strict inequalities for intermediate fields to ensure the cursor correctly "steps" through the results without including duplicates or skipping valid entries.
Go packages for Google Cloud Platform services.
go get cloud.google.com/go/firestore@latest # Replace firestore with the package you want to use.
NOTE: Some of these packages are under development, and may occasionally make backwards-incompatible changes.
For an updated list of all of our released APIs please see our reference docs.
Our libraries are compatible with the two most recent major Go releases, the same policy the Go programming language follows. This means the currently supported versions are:
By default, each client library will use Application Default Credentials (ADC) to automatically configure the credentials used in calling the API endpoint. When using the libraries in a Google Cloud Platform environment such as Compute Engine, Kubernetes Engine, or App Engine, no additional authentication steps are necessary. See Authentication methods at Google and Authenticate for using client libraries for more information.
client, err := storage.NewClient(ctx)
For applications running elsewhere, such as your local development environment, you can use the gcloud auth application-default login command from the Google Cloud CLI to set user credentials in your local filesystem. Application Default Credentials will automatically detect these credentials. See Set up ADC for a local development environment for more information.
Alternately, you may need to provide an explicit path to your credentials. To authenticate using a service account key file, either set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path to your key file, or programmatically pass option.WithCredentialsFile to the NewClient function of the desired package. For example:
client, err := storage.NewClient(ctx, option.WithCredentialsFile("path/to/keyfile.json"))
You can exert even more control over authentication by using the credentials package to create an auth.Credentials. Then pass option.WithAuthCredentials to the NewClient function:
creds, err := credentials.DetectDefault(&credentials.DetectOptions{...})
...
client, err := storage.NewClient(ctx, option.WithAuthCredentials(creds))
Contributions are welcome. Please, see the CONTRIBUTING document for details.
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Contributor Code of Conduct for more information.