I did a little testing and may have found one case that fits what you’re looking for:
I tried a small public FishEye8K probe because you specifically mentioned fisheye, and one case looks fairly close to the “confidently wrong” failure mode you asked for.
The strongest example I found is camera1_A_299.png. I took the native frame and applied a lossless 90° CCW Pillow transpose. Since FishEye8K does not provide roll ground truth, I did not treat native = 0° as ground truth; instead, I used each model format’s native prediction as the reference and checked whether the transformed prediction moved by the expected 90°.
For Fast:
| Fast format |
Equivariance-expected after 90° CCW |
Predicted |
Deviation |
Confidence |
Fast standard |
| INT8 |
269.769° |
91.183° |
178.586° |
0.715 |
answers |
| FP32 |
269.053° |
91.257° |
177.795° |
0.692 |
answers |
| FP16 |
269.052° |
91.253° |
177.799° |
0.693 |
answers |
So Fast is landing almost exactly on the 180° opposite branch, and that branch choice reproduced in the shipped INT8, FP32, and FP16 models.
As a control, Max INT8 on the same transformed image predicted 270.894° with confidence 0.816, only about 0.888° from its own equivariance-expected angle.
That makes this look less like an INT8-only quantization artifact and more like a reproducible Fast-side 180° ambiguity for this particular fisheye scene/orientation. I would not generalize that to fisheye images overall, though.
The operating point also matters quite a bit here:
- Fast standard: answers the opposite branch in all three formats.
- Fast strict: abstains in all three.
- Balanced standard: does not route to Max, so it answers from Fast here.
- Balanced strict: abstains.
- Pro: routes this case to Max.
- Max: lands on the expected branch in the control I tested.
One detail worth noting: FP32/FP16 Balanced-standard are almost exactly on the threshold boundary — confidence is about 0.692, versus a published standard threshold of 0.691 — so I would treat those as boundary cases rather than a large-margin miss.
This seems potentially useful as a small regression/example case because it separates three things reasonably cleanly: the raw Fast branch choice, confidence/abstention, and Fast→Max routing.
Exact reproduction details and output distribution
I pinned the public artifacts rather than testing moving main revisions:
- model: ortusai/rightwayup
- model revision:
71ea0d1c20f6bb973f77351cd0a9202cca98796f
- package:
rightwayup==1.0.0
- source dataset: Voxel51/fisheye8k
- dataset revision:
61edd7ae6d4fabf57454feac0a0dfeadc9e40308
- source image:
data/camera1_A_299.png
- transform: Pillow
Image.Transpose.ROTATE_90
- Fast files:
fast-s224-int8.onnxfast-s224-fp32.onnxfast-s224-fp16.onnx
- Max control:
Pillow documents Image.transpose() as a flip/rotation operation in 90° steps: Pillow Image.transpose() documentation.
I also looked at the raw 360-bin output rather than only the decoded angle. Summing probability mass within ±10° of the equivariance-expected angle and its 180° antipode gave:
| Fast format |
Expected-side mass ±10° |
Antipode mass ±10° |
| INT8 |
0.166 |
0.703 |
| FP32 |
0.185 |
0.680 |
| FP16 |
0.184 |
0.680 |
So this does not look like a near-tie where the final decoded angle barely falls onto the wrong side. For this transformed frame, Fast is putting substantially more mass around the antipodal branch.
I also repeated inference on the exact camera1_A_299 + 90° input. The chosen branch and outputs were effectively deterministic across repeated runs.
Runtime for this confirmation was:
- INT8: ONNX Runtime CPU
- FP32: ONNX Runtime CPU
- FP16: ONNX Runtime CUDA on a T4
The fact that all three select essentially the same branch is the main reason I don’t think INT8 alone explains this case.
Standard / strict / Balanced / Pro on the same case
I used the thresholds from the shipped RightWayUp configuration for the corresponding file format.
For the primary camera1_A_299 + 90° CCW case:
| Format |
Fast conf. |
Fast std. |
Fast strict |
Balanced route threshold |
Balanced std. threshold |
Balanced result |
Pro route threshold |
Pro |
| INT8 |
0.715 |
answer |
abstain |
0.626 |
0.685 |
Fast answers |
0.797 |
routes |
| FP32 |
0.692 |
answer |
abstain |
0.629 |
0.691 |
Fast answers, narrowly |
0.800 |
routes |
| FP16 |
0.693 |
answer |
abstain |
0.629 |
0.691 |
Fast answers, narrowly |
0.800 |
routes |
So I think there are two separate observations here:
- Fast’s raw distribution prefers the opposite 180° branch.
- Whether that becomes a user-visible wrong answer depends on the chosen operating point.
Strict abstention is a useful positive control here: it catches the primary case in every tested Fast format.
Pro is another useful positive control: all three Fast confidences are below its routing threshold, so it sends the case to Max, where my Max INT8 control returned the expected branch.
Balanced standard is particularly interesting as a boundary condition rather than as a broad conclusion. The FP32/FP16 values are only about 0.001–0.002 above the published standard answer threshold.
The exact thresholds are also available in the model repo’s tiers.json: RightWayUp model repository.
Same-camera replication and the broader 18-camera probe
I saw a similar pattern on another frame from the same camera/scene family, camera1_A_200.png, so I would call this replication within camera1 rather than an independent fisheye-domain replication.
For a 90° CCW transpose:
| Format |
Expected |
Fast predicted |
Deviation |
Confidence |
Fast standard |
| INT8 |
269.947° |
91.287° |
178.660° |
0.665 |
answers |
| FP32 |
269.256° |
91.344° |
177.912° |
0.619 |
abstains |
| FP16 |
269.256° |
91.350° |
177.906° |
0.620 |
abstains |
For a 180° transpose:
| Format |
Expected |
Fast predicted |
Deviation |
Confidence |
Fast standard |
| INT8 |
179.947° |
359.873° |
179.925° |
0.698 |
answers |
| FP32 |
179.256° |
359.341° |
179.915° |
0.618 |
abstains |
| FP16 |
179.256° |
359.338° |
179.919° |
0.618 |
abstains |
Max INT8 stayed close to the expected branch on both of those transforms as well.
This second frame is useful because it suggests the branch ambiguity itself is not unique to one exact image, while also showing why confidence calibration matters: the float Fast models still choose the antipodal branch, but standard abstention catches it there.
Before narrowing onto camera1, I also did a small metadata-driven probe using up to two representatives from each FishEye8K camera/location value, preferring time-of-day diversity.
FishEye8K’s card describes 8,000 images from 22 videos and 18 fisheye cameras, and exposes camera/location plus time-of-day metadata: FishEye8K dataset card, source repository.
That panel was 36 images spanning all 18 camera/location values.
Using a >15° quarter-turn equivariance deviation as a review threshold:
- I did not see an answered Max quarter-turn failure in that small panel (
0/36).
- The strongest answered Fast case was the camera1 example above.
- On another camera, I saw a large Fast transformed-angle anomaly where standard abstention did catch it.
So I don’t think the small panel supports a claim like “fisheye breaks RightWayUp” or “Max solves fisheye.” It was mainly useful for finding a reproducible public case and for seeing both failure and successful-abstention behavior.
What I think this does and does not show
What I think the experiment supports fairly directly:
- there is a reproducible ~180° Fast branch switch on this public real-fisheye scene under a 90° quarter-turn;
- it reproduces across the shipped INT8, FP32, and FP16 Fast models;
- therefore the branch choice is not specific to INT8 quantization;
- Max INT8 behaves differently on the same inputs and stays close to the relative expected branch;
- standard/strict and cascade routing materially change whether the ambiguity is exposed as an answer or contained.
What I don’t think this experiment establishes:
- that fisheye projection distortion is the root cause;
- that this generalizes to fisheye cameras overall;
- that Max is generally safe on fisheye;
- that quantization has no effect at all — INT8 does move confidence/probability mass somewhat;
- any absolute FishEye8K rotation accuracy number.
FishEye8K is an object-detection dataset and does not provide roll labels. That is why I treated the native frame only as a reference and used relative quarter-turn equivariance as the stronger check: if the same pixels are transposed by a known 90°, the predicted orientation should move by the corresponding 90° as well.
This is still a synthetic intervention in the sense that I rotated the image rather than physically rotating the camera, but using a 90° Pillow transpose avoids making arbitrary-angle resampling/interpolation the main variable.
There is neighboring camera-calibration work showing that learning-based methods can degrade on fisheye images when the real projection and the projection expected by the method do not match — for example, Wakai et al., “Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion”. I would only treat that as neighboring context, though; this probe does not identify that mechanism inside RightWayUp.
If useful, I can also provide the exact transformed PNGs, the raw per-bin outputs/CSV, or a very small reproduction script for these two camera1 frames.
