-
Notifications
You must be signed in to change notification settings - Fork 29.1k
Transform PointerEvents to the local coordinate system of the event receiver #32192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Thank |
From the test failures, looks like you need to check for a null Also, I've been working on trying to make a Otherwise, we can't nest Listeners (i.e. we can't put a button that handles hover detection inside of another How will this change with this PR? WIll I need to apply a transform at each child layer? (And maybe I should wait until this lands?) |
Actually, in looking at your code some more, it looks like my change was orthogonal to yours, so I committed it: #32350 |
@gspencergoog You're right about the null check. Sorry for being confused. Thanks for looking at it! |
/// Returns the transformation of `position` into the coordinate system | ||
/// described by `transform`. | ||
/// | ||
/// The z-value of `position` is assumed to be 0.0. If `transform` is null, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not just assumed to be zero, it's set to zero (or, really, ignored and zero used instead). And why is that? Wouldn't you want Z to also be preserved if set?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this function is used to convert global to local coordinates, it appears to have the some problem with out of plane rotations as the Transform
widget which I tried to describe here #18408 (comment).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gspencergoog Offsets only have an x and y value - and the z value is always assumed to be zero. The method is just explicitly documenting that.
@spkersten The secret is in the transform matrix that's passed into this. We take the paint transform, set the third row and third column to (0,0,1,0) and then invert it to get a transform matrix that ignores z.
/// Usually, the [global] [Offset] is in the coordinate space of the screen | ||
/// after conversion to logical pixels and the [local] offset is the same | ||
/// [Offset], but transformed to a local coordinate space. | ||
class CombinedOffset { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not a big fan of this name (it's not an Offset subclass)...
Maybe something based on the word Position? I don't have a good idea...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there some way we can avoid making a public class at all for this? It's not clear to me that it does anything outside of this use case we need for this class.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Renamed to OffsetPair
.
This class may be useful for people implementing their own gesture detectors, so I think it is ok to be part of the public API.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've made some comments on the transformations, based on what I found for the Transform
widget's hit testing. Please consider my comments as doubts that could be looked into and not as facts, since I've spend only a limited amount of time with this PR so far.
expect(PointerEvent.transformPosition(Matrix4.identity(), position), position); | ||
final Matrix4 transform = Matrix4.translationValues(10, 20, 0); | ||
expect(PointerEvent.transformPosition(transform, position), const Offset(20.0 + 10.0, 30.0 + 20.0)); | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Having this test with a transformation containing an out of plane rotation (rotateX or rotateY) would be useful as that is a less obvious case that has a bug in the Transform widget.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's the complex "Perspective transform on scrollable" test at the very bottom of the PR that rotates around X and verifies that scrolling still works.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I hope it's okay, but created the test that I proposed to show that it fails: https://github.com/goderbauer/flutter/compare/s2...spkersten:goderbauer-s2-rotation-test?expand=1
Edit: If I replace:
final Matrix4 inverseTransform = Matrix4.inverted(transform);
with
final Matrix4 inverseTransform = Matrix4.inverted(PointerEvent.removePerspectiveTransform(transform));
the test passes.
So it probably comes down to me misunderstanding the code. Played around with some hit testing an transformations and it all seems to work fine. I'm sorry for the confusion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've looked at the "Perspective transform on scrollable" test and it made me really confused. It also fails when I removed the perspective (setEntry(3,2, 0.001)
) and only keep the rotation. So I add the widget tree in an app to see what happens:
class MyApp extends StatelessWidget {
@override
Widget build(BuildContext context) => MaterialApp(home: Foo());
}
class Foo extends StatelessWidget {
@override
Widget build(BuildContext context) {
return Transform(
transform: Matrix4.identity()
..setEntry(3, 2, 0.001)
..rotateX(math.pi / 4)
,
child: Container(
padding: EdgeInsets.all(16),
color: Colors.yellow,
child: Center(
child: Container(
width: 200,
child: ListView.builder(
// controller: controller,
cacheExtent: 0.0,
itemBuilder: (BuildContext context, int index) {
return Container(
height: 100.0,
color: index % 2 == 0 ? Colors.blue : Colors.red,
child: Text('Tile $index'),
);
},
),
),
),
),
);
}
}
I've added the DecoratedBox
to see more clearly what is going on. Here is a screenshot:
So it looks like the children aren't transformed at all!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://github.com/goderbauer/flutter/compare/s2...spkersten:goderbauer-s2-rotation-test?expand=1
Yes, applying MatrixUtils.transformPoint
in both directions will not work as that method completely drops the z value on the floor as you already found out. If you use that method to transform a point P(x, y, 0) on the xy-plane of a x-rotated render plane you will end up with a point P'(x', y', z') and z' will most likely not be 0. If you now take the x' and y' of that point and run it through transformPoint
again (which just sets z to 0), you will not end up with x and y again.
In combination with PointerEvent.removePerspectiveTransform
it works because that method modifies the transform matrix in a way to map point P(x, y, 0) from the xy-plane of the render plane to the point P'(x', y', 0) on the xy-plane of the screen (that's the point on the screen at which P will be drawn). Because z is zero for both points, we can now use MatrixUtils.transformPoint
to correctly transform between the two.
Hope this makes sense? Looking at the other example you posted now...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
About your second comment: That looks like an iOS-specific rendering bug. On Android your example renders fine. Let me do a little investigation and file a separate bug about it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, it looks like it was a bug with the old engine version that this PR was synced to. I just pulled in master and everything renders correctly. (And hit testing also works as expected.)
/// Returns the transformation of `position` into the coordinate system | ||
/// described by `transform`. | ||
/// | ||
/// The z-value of `position` is assumed to be 0.0. If `transform` is null, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this function is used to convert global to local coordinates, it appears to have the some problem with out of plane rotations as the Transform
widget which I tried to describe here #18408 (comment).
/// The resulting matrix can be used to determine the point in the local | ||
/// coordinate system of the transformed component that is exactly under the | ||
/// user's finger when the user is touching the screen. | ||
static Matrix4 paintTransformToPointerEventTransform(Matrix4 paintTransform) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand why this would work without inverting the matrix. Let's say the paint transformation is to scale both x and y by a factor 2: T = diag(2, 2, 1, 1)
. That means pixels of the widget are visually presented twice as far from the origin than in the local coordinate system. Then when a user taps the screen on one such pixel, that tap position will be twice as far from the origin than in the local coordinate system as well. So to do hit testing in the local coordinate system, the point would have to be transformed by diag(0.5, 0.5, 1, 1)
. That should generalise to the inverse of T
.
I see now that the RenderObject
is still responsible for the inversion, so maybe just the name of this function is confusing to me. And the reason why the perspective would have to be removed is unclear to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I renamed this function to make it clearer what it actually does.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I took out the event transformation for mouse-specific events (enter/hover/exit) since supporting those correctly will require some re-structuring of how the MouseTracker works. I filed #33675 to fix that. Please upvote the issue if transformed mouse events are important for you. |
Note: This PR is a follow-up to #31894.
Description
Before this change,
PointerEvent
s would always be expressed in the global coordinate system of the screen. The change addslocalPosition
andlocalDelta
toPointerEvent
s, which are expressed in the local coordinate system of the event receiver. This means - among other things - that scrolling now sticks to your finger even if the Scrollable has been scaled up or down. Furthermore, a rotated scrollable can now be scrolled by moving the finger in the (rotated) direction of the scrollview.Furthermore, this change also fixes hit testing for perspective transforms. For example, if a view is rotated along its x axis, buttons now continue to react to touch events.
Related Issues
Fixes #29916
Fixes #18408
Fixes #31958
Tests
I added the following tests:
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes (
[x]
). This will ensure a smooth and quick review process.///
).flutter analyze --flutter-repo
) does not report any problems on my PR.Breaking Change
Does your PR require Flutter developers to manually update their apps to accommodate your change?