Question

which quantity remains unchanged under coordinate transformations

Answer 1

Answer:

With A, B, and θ all left unchanged by the rotation of the coordinate system, the scalar product remains unchanged, and is therefore invariant. Note that the magnitude-squared of a vector is simply a scalar product of the vector with itself (→A⋅→A=A2), so these two invariants are consistent with each other.