[css-media-queries] Orientation does not respect soft keyboard on mobile devices

[KSCooke]

https://www.w3.org/TR/css3-mediaqueries/#orientation
https://drafts.csswg.org/mediaqueries-3/#orientation

When using @media screen and (orientation: portrait) and @media screen and (orientation: landscape) on a mobile device in portrait mode with the soft keyboard open the orientation is calculated as landscape and not portrait.

I think it would be better if orientation calculation could be based on device-height > device-width so that it would work properly in a mobile environment. If that is not possible then instead to create a device-orientation that could work off of device-height > device-width.

[quasicomputational]

What about a page that has quite a different layout for landscape and portrait? If that's toggled when the user opens the soft keyboard, it could be quite confusing for all involved (e.g., the UA might not be able to maintain scroll position).

[jonjohnjohnson]

Keep in mind we have had the deprecated https://drafts.csswg.org/mediaqueries-4/#device-aspect-ratio which already covers a device-orientation. Simply min-device-aspect-ratio: 1/1 for landscape & max-device-aspect-ratio: 1/1 for portrait. Or if you don't want the "square" cutoff, use any ratio you find better fits your interface. The orientation MQ is just a shorthand for the fuller aspect-ratio MQ.

@quasicomputational I think your question is exactly what the OP is concerned about. Touch implementations of orientation often already change when soft keyboards open/close. https://stackoverflow.com/questions/8883163/css-media-query-soft-keyboard-breaks-css-orientation-rules-alternative-solut

This is sadly a place where media queries cross with the compat around visual and layout viewport. Sometimes I want an MQ that matches the application of soft keyboards, sometimes I don't. Embedded browser projects, like cordova even have work arounds for this lack.

• https://medium.com/samsung-internet-dev/toolbars-keyboards-and-the-viewports-10abcc6c3769
• https://developers.google.com/web/updates/2017/09/visual-viewport-api

And on top of visual and layout viewport issues, the device MQ doesn't nail it when we have iOS Split View and general multitasking interfaces where viewports don't match the devices dimensions anyway.
:(

[ChumpChief]

The orientation MQ should be adhering to the layout viewport width/height - as called out in the Visual Viewport API spec it's an explicit goal for the OSK to avoid altering layout.

It can't be device-* since this won't properly account for windowed environments, split views, multitasking as @jonjohnjohnson calls out. But I think layout viewport probably matches what @KSCooke is describing.

@KSCooke -- where were you observing this behavior (browser/device)?

[css-meeting-bot]

The CSS Working Group just discussed Orientation does not respect soft keyboard on mobile devices.

The full IRC log of that discussion

<heycam> Topic: Orientation does not respect soft keyboard on mobile devices
<heycam> github: https://github.com//issues/3587
<heycam> florian: the problem here is as the title says, when the soft keyboard pops up, the screen effectively switches from portrait to landscape
<heycam> ... but that is not reflected by the orientation media feature
<heycam> ... two ways to look at this
<heycam> ... one is to think first if the soft keyboard is more of an overal, or if it changes the size of the viewport
<heycam> .... currently per spec the width/height media features, aspect-ratio, orientation, as well as vw/vh units or the position of fixed pos elements, all of this is supposed to be in sync with the viewport
<heycam> ... if the keyboard is an overaly none changes, if it changes the viewport, then all should change
<heycam> ... so I think it's clear, the UA can choose if it's an overlay or not
<heycam> ... but maybe orientation maybe could be different
<heycam> ... if the author wants to query the orientation of the device itself
<heycam> ... we have some device-related features that are deprecated
<heycam> ... but we don't have one for device-orientation, it sounds like it might be useful on mobile, unclear on desktop
<heycam> ... whether it's useful I'm worried about changing the definitino of orientation
<heycam> ... since it's in sync with the other ones
<heycam> fantasai: I don't think we should change the definition
<heycam> Rossen: definition of screen orientation should not be changed for sure
<heycam> ... based on size
<heycam> ... either platform specific settings which could apply to any shape of output device regardless of width/height being greater than 1 or the other
<heycam> florian: in general I agree but I wonder what we should do if you're on a phone and the keyboard resizes the screen from portrait to landscape, in a way it's not really a landscape orientation, it's a squished portrait
<heycam> ... but if you have a different design for landscape you don't necessarily want to change to it
<heycam> ... but it's a bit fuzzy
<heycam> Rossen: so you're saying by changing the visual viewport you'll reeval the MQ and it's now landscape?
<heycam> florian: yes per spec
<heycam> ... comparing the width/height of the viewport
<heycam> ... if you choose to have a soft keyboard that goes over the viewport and doesn't resize it that 's fine too
<bkardell_> but does anyone do that?
<heycam> bkardell_: does anyone do a keyboard overlay? on my Android devices it seems to resize the viewport
<heycam> florian: I think everyone does resize. it's a UA choice
<heycam> ... the only thing the spec mandates is taht all these things go together
<heycam> ... no way to distinguish units from the MQs or something like that
<heycam> bkardell_: so it does resize the viewport but it doesn't reclaculate?
<heycam> florian: it should recalculate
<heycam> tantek: I just tried doing this on Safari. a popup select chooser did not resize the viewport
<heycam> florian: did you check if it does different things for MQs and vw/vh units?
<heycam> tantek: no, just trying on some websites
<heycam> ... it seems to overlay the user entry portion on top of the page, doesn't resize the page at all
<heycam> ??: ...
<heycam> jensimmons: this issue will only come up if the site has some CSS to adapt to the viewport size
<heycam> tantek: I wouldn't trust FB as an example, they might have some crazy JS in there
<heycam> jensimmons: would be helpful to have some examples to open up on different phones to test this
<heycam> ... what do we want? consistency in following the spec?
<bkardell_> I mean, they have to know where the viewport is right? the same on twitter btw
<tantek> I tried the popup select on https://www.purpleair.com/
<heycam> ... the issue about the device being portrait/landscape, but what about min-height max-height
<heycam> ... do we need to give authors the choice of whether the keyboard is overlaid? or maybe it doesn't come up enough for us to care?
<heycam> tantek: I just tried a simple textarea on a mediawiki page
<heycam> ... the viewport is also maintined, the keyboard comes up just overlaying
<heycam> ... iOS Safari
<heycam> florian: so the page has something that would resize vertically?
<heycam> ... or it's transparent?
<heycam> tantek: the latter
<jensimmons> We need something that resizes in the vertical direction like this: https://labs.jensimmons.com/2017/01-008C.html **PLUS** a text input
<heycam> florian: this doesn't necessarily give us the answer
<heycam> tantek: for a file input control, it does an overlay of its own UI stuff on top of the page, it doesn't resize anything
<heycam> ... feels like iOS is being consistent
<heycam> florian: unless you have something that is sizing to the veritcal axis, you will not be able to tell between resizing the viewport and not
<heycam> tantek: if we can have a test file
<heycam> jensimmons: I just dropped a demo
<heycam> ... no textarea, but you can see that's an example where if you resize in the vertical direction the layout changes drastically
<heycam> ... the combo of that plus something that triggers the keyboard
<heycam> florian: the test tantek suggested is worth writing
<heycam> ... the other thing: if people have MQs that react to orientation, what are they doing with these?
<heycam> ... so that we can evaluate a bit more whether that is indeed the right thing to do, keeping everything in sync
<heycam> ... or if there is something valid to do that doesn't keep them in sync
<heycam> ... I;m a bit confused what the exact use case is
<heycam> ... I would be uncomfortable with these things getting out of sync, I don't understand what the need is currently
<bkardell_> if you were in a twitter dm and clicked to add text, couldn't the field you were typing in disappear if it overlaid, since it is locked to the bottom of the viewport?
<heycam> myles: if you tap the input do you want the page to relayout?
<heycam> jensimmons: answer is usually no
<heycam> ... but right now it depends on the OS
<heycam> florian: giving the author a choice doesn't sound insane
<heycam> ... what does the MQ evaluate to is a different isuse
<heycam> jensimmons: there are not many people who are making changes to vertical height
<heycam> ... apple is loading different images
<heycam> ... I think this is an issue though
<heycam> ... I'm trying to push designers into understanding
<heycam> ... we'll see in 10 years if I or someone succeeds
<heycam> ... I think this is a case authors should care about
<heycam> florian: responding to the vertical dimension, definitely
<heycam> ... should or should not respond to the screen being shrunk due to the keyboard is harder
<heycam> ... in response to the window resizing to be shorter, and expecitng it not to be relaying out to a different reason (the keyboard) ...
<heycam> Rossen: in all this discussion we've only been considering vertical viewport and not the visual viewport
<heycam> ... if you look atthe last comment, that was added by Matt Rakow
<heycam> ... he has a strong case for taking the visual viewport api spec into consideration with this proposal
<heycam> ... and highlighting the fact that if we start changing what is being proposed here, then windowed / multitasking will start to break
<heycam> ... so we can't really make this to be a device thing
<heycam> florian: is there a right thing to do then?
<heycam> jensimmons: I think it's about visual viewport
<heycam> ... authors think about that
<heycam> florian: there are two points to investigate. one is tantek raised, checking to see that all these things are synchronised
<heycam> ... the other is understanding use cases better
<heycam> ... maybe I can ask Jen to do some homework on the use cases?
<heycam> jensimmons: the use cases [...] if we want to know if we wanted to add something we don't have here, we should think about app use cases
<heycam> florian: I see more of a need for overlay/resizing keyboard, and having the MQs respond to that
<heycam> ... rather than a bunch of MQs that some respond to the viewport and some don't
<tantek> From what I can tell, all input element activations on iOS browser (Safari, Firefox) are overlays, and the page does not resize / reflow / reformat at all. https://www.purpleair.com/map is one example of a page that reflows/reformats to fit the viewport
<myles> can you guys hear me when I talk?
<heycam> Rossen: action item here for jen and ideally others to identify use cases
<tantek> That is, there is no resizing of the viewport on iOS when a soft keyboard, soft popup select, soft file upload UIs are activated by the user
<heycam> ... also we'll hear what Kevin Cook (the issue raiser)
<myles> the iOS keyboard is designed to work in a particular way. There are platform conventions that browsers are designed to obey
<heycam> ... then based on this we might continue the discussion, see what we can do
<heycam> ... sound reasonable?
<heycam> florian: yes
<jensimmons> +1
<heycam> ... I can write a test case and check on Android
<myles> Giving the choice to web authors would be something that would be incompatible with the design of iOS
<heycam> Rossen: I'm more interested in if we can identiy use cases
<heycam> ... not just a test case
<heycam> florian: me too
<heycam> Rossen: but if there is a design pattern blocked/broken by this
<heycam> ... that would gibve a lot more weight to consider this
<tantek> agreed with Rossen's prioritization
<heycam> florian: also understanding what browsers do today is useful

[jensimmons]

We need demos. And use cases. I'll see if I can make one or a few. I won't have much time in the next few weeks — if too much time slips by, do feel free to ping me and remind me to do this in mid-March.

[jensimmons]

I dropped an inquiry on Twitter to see if web designers/developers have desire, use cases or demos:
https://twitter.com/jensimmons/status/1093313475563134977

[KSCooke]

Our current use case is that we have an app with columns of data along the ion-list. When the app goes to landscape we did not want to leave the extra screen real estate unused so we added another column of data to the list allowing for users to have more information. At the top, we have a search bar that allows the user to filter the list. On portrait, this is a very jarring experience when using the orientation media query to show or hide this column.

I attempted to use the min/max device aspect ratio and it did not seem to achieve the goal I was looking for.

[gevgeny]

In or games we use orientation media query in order to set game controls in a corresponding place in mobile devices. When user is holding a mobile device in portrait mode he is (usually but not always) doing it with one hand, thus we display controls in the bottom. When user is holding a device in landscape mode it is comfortable to play with two hand thus controls are in the left and right.

In our "product configurator" usecase, landscape mode on mobile devices brings up an entirely different UI where the client can see the product (which is long and wide) in bigger size. Portrait is otherwise the main editing mode of the application. When the soft keyboard is triggered, the landscape mode UI is shown, a clearly unintended result. There should be something indicating how the device is being held/used that is independent of the keyboard state.

[buitrbao222]

any update on this?

[cvrebert]

Presumably y'all want a media query based on the aspect ratio of the (new-ish) lvh/lvw units.
But personally, it seems like most of the above use-cases ought to simply use a min-width media query instead of trying to base it off of orientation.