in 1.10+, matrix decomposition components aren't always AbstractMatrix

[aplavin]

Tried upgrading Julia version from 1.9 in one of my envs, and noticed this:

julia> using LinearAlgebra

julia> f(a::AbstractMatrix) = sum(a)  # MWE

julia> f(qr(rand(5,5)).Q)
# 1.9:
-1.4572588703587335
# 1.10+:
ERROR: MethodError: no method matching f(::LinearAlgebra.QRCompactWYQ{Float64, Matrix{Float64}, Matrix{Float64}})

qr(A).Q not being an AbstractMatrix is very surprising even in isolation, but especially so given the previous behavior in Julia.
How come it's allowed in Julia 1.x? Seems like the clear and unambiguous definition of breaking change...
From Julia docs:

However, upgrading to the next Stable release will always be possible as each release of Julia v1.x will continue to run code written for earlier versions.

Introduced in #46196 – deliberately, it wasn't just an oversight.

[giordano]

https://docs.julialang.org/en/v1.9/stdlib/LinearAlgebra/#LinearAlgebra.qr

Q matrix can be converted into a regular matrix with Matrix.

[aplavin]

Could you elaborate a bit? Of course, I need Matrix() to convert an AbstractMatrix to a regular Matrix. But how is that related here?

[giordano]

Could you elaborate a bit?

The documentation of Julia v1.9 is saying that you if you want to convert Q to a regular matrix you should use Matrix(Q).

But how is that related here?

It's related because that was the documented way to go already in Julia v1.9.

[aplavin]

convert Q to a regular matrix you should use Matrix(Q)

Note that Q was never a Matrix, and I don't expect it.
But I do expect to be able to pass Q to functions that take AbstractMatrices.

[giordano]

You may be a victim of Hyrum's Law, where you relied on pre-existing unintended and undocumented behaviour.

[aplavin]

you relied on pre-existing unintended and undocumented behaviour

What exactly is not documented? That "returns a matrix" means "AbstractMatrix"?
This is very common in Julia docs, both in terms of return values and argument types. Just clicked on the very first function on the linear algebra page:

tr(M)
Matrix trace. Sums the diagonal elements of M.

Naturally, tr() is defined for AbstractMatrix.
The same is often for "array" in docs = "AbstractArray", both in Julia and in packages.

This is a documented and expected behavior, the linked PR even broke several existing packages. Note that code relying on qr().Q being an AbstractMatrix is most likely to appear in the enduser code, not in packages: one reasonably expects to take a function from some package that works on "matrices" (AbstractMatrix) and apply it to a QR decomposition component.
Actually, LinearAlgebra.tr(qr(A).Q) doesn't work – no need to even go to packages.

If that's not a breaking change, very few things are :)
If such changes are expected in the future as well, should we clarify the stability docs then?

upgrading to the next Stable release will always be possible as each release of Julia v1.x will continue to run code written for earlier versions. * ** *** ****
*: if the code is in a registered package
**: if the relevant codepath is covered by tests
***: if you make sure your tests always pass on nightly
****: if Julia devs don't decide that breaking your code is a minor change and ask you to change it

[mbauman]

@aplavin What were the actual methods you needed and found to be missing? Were they in LinearAlgebra or in your package? I think the best path forward here is to define those as f(Q::AbstractQ) = f(Matrix(Q)). And it would be great to update QR's docs to ensure it doesn't talk about Q as a "matrix".

This was identified as a breaking change right from the get-go and its pros were very deliberately weighed against the estimated costs of the breakage. It's totally fair that you might weigh those pros and cons differently.