Device Orientation and Touch Events

Sparrow Forum » Troubleshooting

Device Orientation and Touch Events

(25 posts) (4 voices)

Started 9 months ago by paul
Latest reply from Daniel

Tags:

paul
Member
G'day, folks!

I'd really appreciate it if someone could try this for me and report back on the observed behaviour. I have a problem with button touches being vertically offset when the device orientation changes. It may well be touches in general, but buttons are easy to test on.

If you have the Sparrow demo, you'll need to make these changes to run my test. All of these changes are in Game.m:
// Add this to the end of initWithWidth: height: (inside the curly braces and definitely before return self) [[UIDevice currentDevice] beginGeneratingDeviceOrientationNotifications]; [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(orientationChanged:) name:@"UIDeviceOrientationDidChangeNotification" object:nil]; // Add this function below initWithWidth: height: - (void)orientationChanged:(UIEvent *)event { UIDeviceOrientation orientation = [UIDevice currentDevice].orientation; if (orientation == UIDeviceOrientationPortrait) { mMainMenu.rotation = 0; mMainMenu.x = mMainMenu.y = 0; } else if (orientation == UIDeviceOrientationPortraitUpsideDown) { mMainMenu.rotation = SP_D2R(180); mMainMenu.x = self.width; mMainMenu.y = self.height; } NSLog(@"Device Orientation: %i", orientation); }
Note that this needs to be tested on a device. I have only tested it on an iPhone 4 and a 3GS. Now load the demo up and hold it in Portrait orientation. Try to press the "RenderTexture" button on the main menu, but focus on pressing towards the lower half of the button. It should be easy to press and respond as expected. Press the "back" button to return to the main menu.

Now turn the device to PortraitUpsideDown orientation. Again, try to press the "RenderTexture" button focusing on its lower half. If you're like me, it doesn't register the touch until you move your finger upwards towards the top half of the button.

I'd love to solve this little problem because it's important that my game can be played from multiple orientations. It's interesting to note that it doesn't seem to occur in the simulator. Thanks to any and all who spend the time to experiment for me
Posted 9 months ago #
paul
Member

Sorry for bumping, but if I can twist anyone's arm to try this, it would really help It will let me narrow down whether I need to look into Sparrow or my own methods.

Posted 9 months ago #
Daniel
Primary Feather

You're right, I can reproduce it on my iPhone 3GS! Very odd. I don't know yet what causes it ...!

Posted 9 months ago #
paul
Member

Thanks for taking the time to test for me, Daniel! I'll continue to look for the cause of this issue, but please keep me posted on any ideas you might have for the solution. I need to fix this problem.

Posted 9 months ago #
paul
Member

Note that this problem exists for landscape orientations, too. LandscapeLeft responds as expected, whereas LandscapeRight suffers from the vertical offset problem.

Posted 9 months ago #
Daniel
Primary Feather

Hm, when I print out the coordinates I get from UIKit (in SPView.m, 'processTouchEvent:') and touch the top of the screen, I get negative coordinates. I guess this is exactly the area where the menu bar is normally located.

Now that I think about it, this was reported once before in the forum, but the user said he found a workaround, and then I never investigated it.

I can't do anything else today, but I'll see what I can do in the next days. If you want to investigate it yourself, everything relevant should be in the touch processing code of SPView. When we get rid of those negative coordinates, the problem should be solved. I guess.

Posted 9 months ago #
paul
Member
I don't have a solution yet, but I do have a potential workaround that shouldn't affect most people's code. In SPView's setup method, set the layer's bounds before adjusting its properties.
// Add this... self.layer.bounds = CGRectMake(0, 8, self.frame.size.width, self.frame.size.height); // ...before this CAEAGLLayer *eaglLayer = (CAEAGLLayer *)self.layer; ...
But we'd need to establish that an ~8 pixel offset is consistent across devices (ie that it is specific to Sparrow). It's also important that you set the device's "Initial interface orientation" to "Portrait (bottom home button)" in the game's info.plist. This doesn't mean you have to have a portrait-oriented game (and there may be an extended workaround that can avoid this restriction), but it needs to be set to start up in that orientation.

Edit:

Probably doesn't matter, but might be safer to do:
CAEAGLLayer *eaglLayer = (CAEAGLLayer *)self.layer; eaglLayer.bounds = CGRectMake(0, 8, eaglLayer.bounds.size.width, eaglLayer.bounds.size.height);
Posted 9 months ago #
enbr
Moderator

Just for kicks, will you grab the BEDisplayDebugger extentension and upload a few screenshots using it? I have a feeling the problem is not what it appears to be.

Posted 9 months ago #
paul
Member

Enbr, do you get similar results with the demo? If so, it'd probably be easier for you to verify your theory on the device.

Posted 9 months ago #
Daniel
Primary Feather

Thanks for posting the workaround! I will investigate this further to find out what's the root of the problem. I created an issue on the GitHub page so that it's not forgotten.

Posted 9 months ago #
enbr
Moderator

After testing out the bug with BEDisplayDebugger, I would agree that this is definitely a Sparrow issue. I thought at first that it might be about sprite placement.

Posted 9 months ago #
Daniel
Primary Feather

OK, I finally found the time to have a look at this issue.

I created a (Non-Sparrow!) standard view-based application in Xcode, and printed out the coordinates of touch events. This crude illustration (sorry for the quality) contains my findings:

http://img17.imageshack.us/img17/8263/iosorientation.jpg

On the left you find the coordinates of both the topmost and bottommost touches I could make, and their position on the screen, in 2 orientations (portrait and upside-down) and on 2 devices (my iPhone 3GS and my iPad 1).

As you can see, you can't get to the bottom on an upside-down device, no matter which orientation you're using! In portrait mode, however, you can reach (as Paul already wrote) negative coordinates, outside of the screen, at the top.

After this finding, I had a close look at where a touch is registered (by drawing it on the screen) when I touch the screen with my stylus. It's shown in the bottommost illustrations: The point appears slightly upwards to where I touch!

The right column contains my tests with Sparrow. In portrait mode, everything works just like in UIKit, that's why I didn't draw that down. In upside-down mode, though, the coordinates are wrong (shown on the left), because it uses the coordinates that UIKit produces -- which are *not* rotated. You can clearly see that the touch point now appears further down instead of further up.

The right image shows what would have to be done to mimic the UIKit behaviour: we have to subtract "16" from the y-coordinate of the touch. This makes it behave just like UIKit.

My conclusion is that this is exactly what UIKit does internally when you rotate the device. It's clearly visible from the illustrations that the touch sensor is not directly above the screen, but is shifted a little upwards. I guess that Apple made a lot of usability tests and found out that users tend to "feel" their touch point at a position that is slightly further down than it really is, and decided to compensate for that, so that it "feels" right. Even if it isn't.

The UIKit classes are aware of this coordinate-shift, and compensate for that. A rotated Sparrow object is not aware of this, of course, because the UIView it gets its coordinates from is not aware of the rotation.

Unfortunately I haven't checked landscape modes; I'll do so later today, but I expect the outcome to be similar (a shift of +-8 pixels).

In the end, however, this means that the rotation stuff has to be built into Sparrow. Sparrow has to know which orientation you (the developer) are currently working with, so that it can update the coordinates correspondingly.

What do you say? Does that make sense? Or am I drawing wrong conclusions here? I'd be happy about any comments! =)

Posted 9 months ago #
paul
Member

I had similar thoughts on Apple adjusting it for usability concerns, but why would this not be documented somewhere? Or is it documented somewhere?

In any case, I would be happy if I can get the full range in all orientations and for the volume GUI (and any other system GUIs) to be presented in the correct orientation.

Thanks for looking at this, Daniel!

Posted 9 months ago #
Daniel
Primary Feather

Yes, it's odd that it isn't documented -- at least I did not find anything about it!
I'll report back when I have checked the landscape orientations.

Posted 9 months ago #
Daniel
Primary Feather

As I expected: it's the same in landscape modes.

The touch point is registered a little bit above where the finger/stylus hits the screen (8 pixels), and this allows you to register touch events from y=1 (top of the screen) to y=312 (8 pixels above bottom of the screen). You can go from x=1 to x=480 horizontally, though.

Thus, in the landscape modes, Sparrow would need to add / subtract 8 pixels from the x-value, and add 8 pixels to the y-value (in portrait UIKit coordinates) to have Sparrow react exactly like UIKit.

And for now, Paul, you'll have to do that manually in SPView.m, until I find the time to add that change to Sparrow.

A quick fix would be to simply add 8 to the y-value in all orientations. That way, the touch point would simply be where it *really* happens. I don't think anybody would notice that.

Posted 9 months ago #
paul
Member

Daniel, how does that regain the 8 pixels that are lost in landscape? Or do you not experience a loss of 8 pixels worth of touch as I do?

Posted 9 months ago #
Daniel
Primary Feather

Yes, in UIKit, there is always one row, 8 pixels high, that is lost.

The loss of the 8 pixels comes from the fact that UIKit always moves the touch coordinates upwards 8 pixels, and since the touch screen is just as big as the screen, you lose those 8 pixels at the bottom.

The last workaround I described -- adding 8 to the y-value in all orientations -- is the only way to get rid of that, because then, you register the touch at exactly the position it occurs. Thus, the whole touch sensitive screen area can be used.

Posted 9 months ago #
paul
Member

That makes sense to me for Landscape left, because it goes from -8 to 312, but Landscape right goes from 8 to 320. So it doesn't matter how much I add or subtract from that range, it is still only a range of 312. Do you get similar results? Do I just need some sleep?

Edited:

I should clarify that by Landscape right I mean "Landscape (right home button)" in an Info.plist file, which for some strange reason seems to correspond to "UIDeviceOrientationLandscapeLeft"...

Posted 9 months ago #
Daniel
Primary Feather

Hm, think about it that way: no matter the orientation of the device, we want the SPView to be in normal portrait mode, because that's the optimal orientation for OpenGL. We only rotate the content with Sparrow.

Now, if you look at the top left illustration I posted (UIKit in portrait mode), then you see that y ranges from -8 to +472. If we add 8 to that, we've got 0 to 480 -- that's what we want. And if the SPView is always in portrait mode, that will be the same in all orientations.

Or is it me who's missing something?

Posted 9 months ago #
Daniel
Primary Feather
No, sorry, it was indeed me who was wrong.

I had another look at it, and the 8 pixel offset is not dependent on the orientation of the UIView, but on the status bar orientation (!).

So you can view it that way: the touch coordinates always want to get close to the menu bar, and move 8 pixels in its direction. (Even if it's not visible, it's the setting that counts.)

That can be changed with with:
[[UIApplication sharedApplication] setStatusBarOrientation:UIInterfaceOrientationXYZ];
So, in the end you're right: you're losing an 8 pixel bar in all orientations. But that's the case in all iPhone applications, and since almost nobody ever noticed, I don't think it's problematic!

But it tells us one important lesson: when rotating the Sparrow content, always rotate the status bar as well, even if it's not visible. Then, all touch coordinates will work as expected!
Posted 9 months ago #
Daniel
Primary Feather

I created a tutorial on the Sparrow wiki that summarizes what has to be done to automatically rotate the interface *and* still have touch events that feel correct.

I did not mention the 8 pixels, though. I don't want to confuse the readers.

Posted 9 months ago #
Doolali
Member

Any chance of being able to fix the offset in the Sparrow API?

Posted 9 months ago #
Daniel
Primary Feather

Sorry, I haven't been clear on this: the fix is not necessary! When you set the status bar correctly, the touch coordinates are just as expected.

Posted 9 months ago #
paul
Member

I think that solution is workable. If people are used to their device operating in this way, then who am I to change it. But I find it most unusual, and I consider it to be less desirable for games than it may be for UIKit applications.

Just to frame this discussion with my results:

With the CAEAGLLayer "fix" (offsetting the layer's origin by +8 on the x-axis), I can get a full 320 points of touch if the device is in Landscape (left home button) and the status bar is set to the same orientation as the device, but I cannot get any more than 312 points if the device is in Landscape (right home button) with the status bar also set to the same orientation as the device.

The reason this operates in an unusual manner is that you can reach the top of the view in Landscape (right home button), but horizontal movement beyond that point is still reported. So your touch is progressing horizontally (despite the fact that you've moved your touch further up than is the reported limit of touch in the vertical direction), but it is not progressing vertically! This same problem does not occur in Landscape (left home button) because you can go beyond the limit into the negatives (with the "fix" you don't even need to go into the negatives to get a full 320 points). So movement touches (like if you're dragging or rotating something and just want a vector for direction) report further in one orientation than in the opposite one, resulting in distorted vectors.

It makes me think there is a transform that is missing between the UITouch and the SPTouch, but that is pure speculation. I notice that another unnamed engine seems to set the status bar to the opposite orientation of the device for landscape orientations only. You can see this in CCDirectorIOS.m. I don't have anymore experience with this than a cursory glance at some source code, so there may be another explanation for that.

Edited:

You could clamp it to 0 to even out both sides, but at some point it begins to smell of hack. Also, the reason for setting the status bar to the opposite orientation of the device is that (for some special/strange reason) UIDeviceOrientationLandscapeX corresponds to UIInterfaceOrientationLandscapeY.

Posted 9 months ago #
Daniel
Primary Feather

Thanks for summarizing it, Paul!

I, too, think that this is a little weird. But given the fact that it now operates the same way in Sparrow and UIKit (with the status bar at the correct place), I think I'll leave it at that, for now.

Posted 9 months ago #

RSS feed for this topic

Reply

You must log in to post.