summaryrefslogtreecommitdiff
path: root/django/db/models/sql/compiler.py
AgeCommit message (Collapse)Author
2026-02-03[5.2.x] Fixed CVE-2026-1312 -- Protected order_by() from SQL injection via ↵Jacob Walls
aliases with periods. Before, `order_by()` treated a period in a field name as a sign that it was requested via `.extra(order_by=...)` and thus should be passed through as raw table and column names, even if `extra()` was not used. Since periods are permitted in aliases, this meant user-controlled aliases could force the `order_by()` clause to resolve to a raw table and column pair instead of the actual target field for the alias. In practice, only `FilteredRelation` was affected, as the other expressions we tested, e.g. `F`, aggressively optimize away the ordering expressions into ordinal positions, e.g. ORDER BY 2, instead of ORDER BY "table".column. Thanks Solomon Kebede for the report, and Simon Charette and Jake Howard for reviews. Backport of 69065ca869b0970dff8fdd8fafb390bf8b3bf222 from main.
2025-04-07[5.2.x] Fixed #36301 -- Fixed select_for_update(of) crash when using ↵Simon Charette
values()/values_list(). Regression in 65ad4ade74dc9208b9d686a451cd6045df0c9c3a which allowed for annotations to be SELECT'ed before model field references through values()/values_list() and broke assumptions the select_for_update(of) table infererence logic had about model fields always being first. Refs #28900. Thanks OutOfFocus4 for the report and Sarah for the test. Backport of 71a19a0e475165dbc14c1fe02f552013ee670e4c from main
2025-02-15[5.2.x] Refs #36181 -- Removed the obsolete SubqueryConstraint machinery.Mariusz Felisiak
Adding proper support for subquery right-hand-sides to TupleIn made it obsolete. Backport of d386405e04dac50656af50d100a14efdf8c58e8f from main Co-authored-by: Simon Charette <charette.s@gmail.com>
2025-01-28[5.2.x] Fixed #36122 -- Raised FieldError when updating with composite ↵Simon Charette
reference value. Thanks Jacob Walls for the report and test. Backport of efec74b90868c2e611f863bf4301d92ce08067e8 from main.
2025-01-13Fixed #36086 -- Fixed crash when using GeneratedField with non-AutoField pk.Simon Charette
The previous logic was systematically attempting to retrieve last_insert_id even for models without an AutoField primary key when they had a GeneratedField on backends that can't return columns from INSERT. The issue affected MySQL, SQLite < 3.35, and Oracle when the use_returning_into option was disabled and could result in either crashes when the non-auto primary key wasn't an IntegerField subclass or silent misassignment of bogus insert ids (0 or the previous auto primary key insert value) to the first defined generated field value.
2025-01-13Refs #373 -- Removed unused composite pk code in SQLInsertCompiler.Simon Charette
This logic could only be exercised if the composite primary key included an AutoField but it's not allowed yet (refs #35957). It was also slightly broken as it expected the AutoField to always be the first member of returning_fields.
2025-01-08Fixed #36065 -- Fixed ordering by expression referencing composite primary key.Simon Charette
Thanks Jacob Walls for the report and test and Csirmaz Bendegúz for the review.
2025-01-03Fixed #35918 -- Added support for execute_sql to directly return row counts.Raphael Gaschignard
2024-12-11Refs #35936 -- Avoided field placeholder lookup for each value inserted.Simon Charette
By building the list of placeholders for each inserted fields once it doesn't have to be looked up for each inserted rows twice. The query_values_10000.benchmark.QueryValues10000.time_query_values_10000 ASV benchmark showed a 5% speed up for 10k items on SQLite for a single field insertion. Larger performance gains are expected when more fields are involved.
2024-11-29Fixed #373 -- Added CompositePrimaryKey.Bendeguz Csirmaz
Thanks Lily Foote and Simon Charette for reviews and mentoring this Google Summer of Code 2024 project. Co-authored-by: Simon Charette <charette.s@gmail.com> Co-authored-by: Lily Foote <code@lilyf.org>
2024-08-02Fixed #35643 -- Fixed a crash when ordering a QuerySet by a reference ↵Simon Charette
containing "__". Regression in b0ad41198b3e333f57351e3fce5a1fb47f23f376. Refs #34013. The initial logic did not consider that annotation aliases can include lookup or transform separators. Thanks Gert Van Gool for the report and Mariusz Felisiak for the review.
2024-07-22Fixed #35614 -- Prevented SQLCompiler.as_subquery_condition() from mutating ↵Bendeguz Csirmaz
a query.
2024-07-19Fixed #35559 -- Avoided unnecessary query on sliced union of empty queries.Simon Charette
While refs #34125 focused on the SQL correctness of slicing of union of potentially empty queries it missed an optimization opportunity to avoid performing a query at all when all queries are empty. Thanks Lucidiot for the report.
2024-07-16Doc'd purpose of tuple() in SQLCompiler.get_from_clause().Tim Graham
It was added in 01d440fa1e6b5c62acfa8b3fde43dfa1505f93c6 to prevent "RuntimeError: OrderedDict mutated during iteration". That particular issue was fixed in d660cee5bc68b597503c2a16f3d9928d52f93fb4 but the issue could remain in Join.as_sql() subclasses. Co-authored-by: Simon Charette <charette.s@gmail.com>
2024-07-15Removed leftover KeyError handling after Query.tables attribute cleanup.nessita
Follow up from f7f5edd50d03e8482f8a6da5fb5202b895d68cd6.
2024-07-03Fixed #28900 -- Propagated all selected fields to combinator queries.Simon Charette
Previously, only the selected column aliases would be propagated and annotations were ignored.
2024-07-03Refs #28900 -- Made SELECT respect the order specified by values(*selected).Simon Charette
Previously the order was always extra_fields + model_fields + annotations with respective local ordering inferred from the insertion order of *selected. This commits introduces a new `Query.selected` propery that keeps tracks of the global select order as specified by on values assignment. This is crucial feature to allow the combination of queries mixing annotations and table references. It also allows the removal of the re-ordering shenanigans perform by ValuesListIterable in order to re-map the tuples returned from the database backend to the order specified by values_list() as they'll be in the right order at query compilation time. Refs #28553 as the initially reported issue that was only partially fixed for annotations by d6b6e5d0fd4e6b6d0183b4cf6e4bd4f9afc7bf67. Thanks Mariusz Felisiak and Sarah Boyce for review.
2024-04-23Refs #35356 -- Clarified select related with masked field logic.Simon Charette
By always including related objects in the select mask via adjusting the defer logic (_get_defer_select_mask()), it becomes possible for select_related_descend() to treat forward and reverse relationships indistinctively. This work also simplifies and adds comments to select_related_descend() to make it easier to understand.
2024-04-23Fixed #35356 -- Deferred self-referential foreign key fields adequately.Simon Charette
While refs #34612 surfaced issues with reverse one-to-one fields deferrals, it missed that switching to storing remote fields would break self-referential relationships. This change switches to storing related objects in the select mask instead of remote fields to prevent collisions when dealing with self-referential relationships that might have a different directional mask. Despite fixing #21204 introduced a crash under some self-referential deferral conditions, it was simply not working even before that as it aggregated the sets of deferred fields by model. Thanks Joshua van Besouw for the report and Mariusz Felisiak for the review.
2024-03-15Fixed #35294 -- Fixed TEXT format of QuerySet.explain() for long plans.Adam Johnson
co-authored-by: Gordon <gordon.wrigley@gmail.com> co-authored-by: Simon Charette <charette.s@gmail.com>
2024-02-26Fixed #35241 -- Cached model's full parent list.Adam Johnson
co-authored-by: Keryn Knight <keryn@kerynknight.com> co-authored-by: Natalia <124304+nessita@users.noreply.github.com> co-authored-by: David Smith <smithdc@gmail.com> co-authored-by: Paolo Melchiorre <paolo@melchiorre.org>
2024-01-26Applied Black's 2024 stable style.Mariusz Felisiak
https://github.com/psf/black/releases/tag/24.1.0
2023-12-12Fixed #34013 -- Added QuerySet.order_by() support for annotation transforms.Simon Charette
Thanks Eugene Morozov and Ben Nace for the reports.
2023-12-07Fixed #35024 -- Fixed model instance creation crash on ↵Mariusz Felisiak
GeneratedField.output_field with backend converters. Regression in d9de74141e8a920940f1b91ed0a3ccb835b55729. This is a long standing issue, however it caused a crash of GeneratedFields for all output fields that have backend-specific converters when the RETURNING clause is not supported (MySQL and SQLite < 3.35). That's why severity was exacerbated.
2023-11-13Fixed #34967 -- Fixed queryset crash when grouping by constants on SQLite < ↵David Sanders
3.39. On SQLite < 3.39, this forces a GROUP BY clause with a HAVING clause when no grouping is specified. Co-authored-by: Simon Charette <charette.s@gmail.com>
2023-05-22Fixed #34580 -- Avoided unnecessary computation of selected expressions in ↵Simon Charette
SQLCompiler. Performance regression in 278881e37619278789942513916acafaa88d26f3. Co-authored-by: David Smith <smithdc@gmail.com>
2023-05-01Fixed #33759 -- Avoided unnecessary subquery in QuerySet.delete() with ↵4the4ryushin
self-referential subqueries if supported.
2023-02-27Fixed #34368 -- Made subquery raise NotSupportedError when referencing outer ↵Simon Charette
window expression. Regression in f387d024fc75569d2a4a338bfda76cc2f328f627. Co-authored-by: Jannis Vajen <jvajen@gmail.com>
2023-02-27Fixed #34372 -- Fixed queryset crash on order by aggregation using OrderBy.Simon Charette
Regression in 278881e37619278789942513916acafaa88d26f3 caused by a lack of expression copying when an OrderBy expression is explicitly provided. Thanks Jannis Vajen for the report and regression test.
2023-02-20Fixed #34346 -- Ordered selected expressions by position.Simon Charette
Used the same approach as for #34176 by using selected expressions position to prevent ambiguous aliases in collisions. Thanks henribru for the report. Regression in 04518e310d4552ff7595a34f5a7f93487d78a406.
2023-02-18Refs #34176 -- Adjusted group by position variables naming to follow SQL spec.Simon Charette
This avoids conceptual collisions with the notion of indices.
2023-01-24Fixed #34227 -- Fixed QuerySet.select_related() with multi-level ↵朱穆穆
FilteredRelation.
2023-01-20Fixed #34267 -- Fixed sliced QuerySet.union() crash.Francesco Panico
Regression in 3d734c09ff0138441dfe0a59010435871d17950f. Thanks Raphaël Stefanini for the report.
2023-01-09Fixed #34176 -- Fixed grouping by ambiguous aliases.Simon Charette
Regression in b7b28c7c189615543218e81319473888bc46d831. Refs #31377. Thanks Shai Berger for the report and reviews. test_aggregation_subquery_annotation_values_collision() has been updated as queries that are explicitly grouped by a subquery should always be grouped by it and not its outer columns even if its alias collides with referenced table columns. This was not possible to accomplish at the time 10866a10 landed because we didn't have compiler level handling of colliding aliases.
2023-01-04Simplified SQLCompiler.get_group_by() a bit.Simon Charette
2022-12-27Fixed #34226 -- Fixed QuerySet.select_related() with multiple ↵朱穆穆
FilteredRelations to the OneToOneField.
2022-12-24Refs #34226 -- Renamed local field variables in ↵Mariusz Felisiak
SQLCompiler.get_related_selections() to avoid redefinition.
2022-12-01Refs #33308 -- Deprecated support for passing encoded JSON string literals ↵Simon Charette
to JSONField & co. JSON should be provided as literal Python objects an not in their encoded string literal forms.
2022-11-22Fixed #34171 -- Fixed QuerySet.bulk_create() on fields with db_column in ↵DevilsAutumn
unique_fields/update_fields. Bug in 0f6946495a8ec955b471ca1baaf408ceb53d4796. Thanks Joshua Brooks for the report.
2022-11-15Fixed #34123 -- Fixed combinator order by alias when using select_related().Simon Charette
Regression in c58a8acd413ccc992dd30afd98ed900897e1f719. Thanks to Shai Berger for the report and tests. Co-Authored-By: David Sanders <shang.xiao.sanders@gmail.com>
2022-11-15Avoided unnecessary usage of RawSQL.Simon Charette
This ensures proper alias quoting.
2022-11-07Refs #33374 -- Adjusted full match condition handling.Simon Charette
Adjusting WhereNode.as_sql() to raise an exception when encoutering a full match just like with empty matches ensures that all case are explicitly handled.
2022-11-07Refs #17144 -- Removed support for grouping by primary key.Simon Charette
No core backend require the feature anymore as it was only added to support a MySQL'ism that has been deprecated since then.
2022-11-07Fixed #31331 -- Switched MySQL to group by selected primary keys.Simon Charette
MySQL 5.7.15 supports group by functional dependences so there is no need to special case group by main table primary key anymore and special case the ONLY_FULL_GROUP_BY sql mode.
2022-10-31Used more augmented assignment statements.Nick Pope
Identified using the following command: $ git grep -I '\(\<[_a-zA-Z0-9]\+\>\) *= *\1 *[-+/*^%&|<>@]'
2022-10-29Fixed #34125 -- Fixed sliced QuerySet.union() crash on a single non-empty ↵Simon Charette
queryset. The bug existed since sliced query union was added but was elevated to query union slices by moving the .exists() optimization to the compiler in 3d734c09ff0138441dfe0a59010435871d17950f. Thanks Stefan Hammer for the report.
2022-10-28Used Query.is_sliced in SQLCompiler.as_sql().Simon Charette
2022-10-18Fixed #34105 -- Fixed crash of ordering by nested selected expression.Simon Charette
This stops ordering by nested selected references. It's not supported on PostgreSQL and not required to support psycopg3. Regression in 04518e310d4552ff7595a34f5a7f93487d78a406. Thanks Matt Westcott for the report.
2022-10-06Refs #31150 -- Enabled implicit GROUP BY aliases.Simon Charette
This ensures implicit grouping from aggregate function annotations groups by uncollapsed selected aliases if supported. The feature is disabled on Oracle because it doesn't support it.
2022-10-06Refs #33992 -- Refactored subquery grouping logic.Simon Charette
This required moving the combined queries slicing logic to the compiler in order to allow Query.exists() to be called at expression resolving time. It allowed for Query.exists() to be called at Exists() initialization time and thus ensured that get_group_by_cols() was operating on the terminal representation of the query that only has a single column selected.
generated by cgit v1.3 (git 2.53.0) at 2026-05-01 22:04:02 +0000