InteractiveViewer, introduced in #56409, should allow an overscroll feature in both directions when its boundary is reached. Google Photos and Apple Photos both have this feature after zooming in slightly on a photo. This is also mentioned in #20175.

Edit: Also, the physics involved should probably be configurable, similar to how it's possible to pass in scroll physics to a scroll view.