[mediaqueries-5] Detect physical keyboard (keyboard capable of shortcuts?)

[birtles]

From stackoverflow: https://stackoverflow.com/questions/36976117/css-media-query-for-whether-a-physical-keyboard-is-attached

Use case: I want show keyboard shortcut labels in my menu / tooltips but only when the user has a physical keyboard.

Rationale (taken directly from the above SO issue):

• nobody would use a soft-keyboard this way even if they could
• the visual display of the keyboard shortcut takes up valuable real-estate
• it's perhaps confusing, or at least inelegant, to display keyboard shortcuts in a keyboardless context

Apparently some people are doing horrible hacks like using @media (hover: hover) and (pointer: fine) { ... } to detect if there is a mouse/trackpad and assuming that means there is a keyboard too.

[tabatkins]

That seems like a pretty reasonable use-case, and I know we thought about whether to expose more input devices.

Thoughts, @frivoal? I'm inclined to add this.

[frivoal]

I've long thought we should be adding something along these lines, without being exactly sure about what we need to add:

• do we need to distinguish between keyboards with and without a numpad?
• do we need to distinguish between computer keyboards and feature phone keyboard that only have a numpad and a handful of extra buttons? How do you classify less-than-full-but-more-than phone things like this: https://web.archive.org/web/20180806062750/https://icdn4.digitaltrends.com/image/blackberry-passport-simple-720x720.jpg?ver=1.jpg
• what about things like gamepads that aren't keyboards but do have buttons? If gamepads aren't keyboards, what is this: https://images.techhive.com/images/idge/imported/imageapi/2014/06/27/21/igrip-100351419-gallery.idge.jpg ? based on what criteria?
• distinguish between keyboard with or without a tab key? with or without arrow keys?
• This probably counts as a physical keyboard rather than on virtual one, on account of not being on screen. But are we sure? That probably depends on the use cases: https://images.techhive.com/images/idge/imported/imageapi/2014/06/27/21/virtual_keyboard-100351425-gallery.idge.jpg
• Do we distinguish between keyboard layouts? Maybe we don't need to distinguish between qwerty and azerty layout since they can type the same characters (mostly), but we do need to distinguish between keyboards that can type ascii letters vs those that can only type other stuff, like https://www.dsi-keyboards.com/wp-content/uploads/2015/03/KA-Thai-20868.jpg
• Do chorded keyboards fall into any of the categories we're otherwise deciding on? if not, does it mean we got them wrong, or that we don't care about such devices?
• in the case of software keyboard, do we distinguish between those that reduce the viewport size when the pop up, and those that get overlaid over the viewport?
• Do we need the primary / non primary input distinction?

I don't have good answers to these questions.

To be clear, I don't mean that we need to have a super rich set of media queries to distinguish between everything. I do mean that the reality of input devices is extremely varied, and that how much we want to expose, and along which axises is not an obvious question. The answers will depend on which set of use cases we want to address. We have a few identified use cases but overall we haven't yet tried to do a broad survey of what people were trying to achieve with these.

Related reading:

• Request: media query feature for device when a virtual keyboard is open/closed #621
  • https://bugzilla.mozilla.org/show_bug.cgi?id=787444
  • https://lists.w3.org/Archives/Public/www-style/2015Apr/0052.html
  • https://discourse.wicg.io/t/onscreen-keyboard-api-to-obtain-dimensions-presence-of-the-keyboard/674
  • https://discourse.wicg.io/t/exposing-a-users-input-modality/1067
• [mediaqueries-5] detect keyboard navigation abilities #1911

[birtles]

For this use case I think all we're interested in is whether or not the user is likely able to use a keyboard for shortcut keys. There may be some keyboard layouts that don't support certain shortcut keys (e.g. keyboards without a function key row) but handling that seems like a separate problem (as does tailoring the shortcut key descriptions to match platform conventions).

Even if the shortcut key is just a you're not likely to use it as a shortcut key on a virtual keyboard, but you are on a physical keyboard.

[samhed]

Another use case that is relevant is simply if there is a physical keyboard available or not. On devices where no physical keyboard is available, our web app wants to show a button that opens a virtual keyboard (by moving focus to a textbox).

As mentioned in the initial post, until we have something like this, we're forced to make the bad assumption that touch devices have no physical keyboards (we use any-pointer: coarse).

[emilio]

Strawman proposal, wdyt of: @media (keyboard: physical|virtual|none)?

[emilio]

If we need further distinctions we can add them in the future, but that should allow most use-cases without raising a lot of hard-to-answer questions?

[tabatkins]

Yup, I think we can extend it later to allow, say, (keyboard: physical numpad) if that's a distinction we want to expose.

When would you have keyboard: none?

[emilio]

Printing perhaps? Other non-interactive media?

[tabatkins]

Yup, printing definitely makes sense, just wanted to check.

[birtles]

Some extra background regarding the motivation and timing for this, Chrome is switching to desktop mode for premium tablets which is going to cause trouble for sites using the UA string to determine if the device has a keyboard or not.

From the article:

it is a good time to confirm that all interactions with the desktop versions of your site work well for users with and without an attached physical keyboard

In my experience, you currently need to use the UA string to avoid false negatives on some devices.

[Crissov]

Media Queries are not intended to be used with CSS only. If they were limited to styling, I’d agree that there should only be minimal support for keyboard detection.
If the displaying keyboard (shortcut) hints use case is valid, MQ would need to be powerful enough to distinguish different types of keyboards and layouts.

For the presence of a keyboard, MQ should distinguish between always and on demand, but perhaps more importantly between separate from the viewport and semi-transparently overlaying, opaquely obstructing and intrusively shrinking it.
For virtual keyboards in particular, there may be reasonable desires to know whether it is used by swiping or typing/pressing and whether 1 finger or thumb, 2 fingers/thumbs from separate hands or all fingers 1 from 1 or 2 hands are typically used. (Key chaining is a bit exotic in alphabetic cultures, but may abhustend describe Asian IMEs in particular.)

For some keys, it could be enough to provide access to locale or system-specific labels elsewhere, e.g. Cmd (Mac) vs. Ctrl (EN) vs. Strg (DE). That is probably outside the scope of both, CSS and MQ (except in env() perhaps).

The exact availability of key group levels like shift [lock] and alt/option, perhaps does not need to be exposed to authors.

Authors would probably want to know sometimes whether the keyboard provides 1D sequential navigation of interaction focus (e.g. with tab) or 2D spatial viewpoint navigation (with 4 to 8 arrow keys, d-pad or joystick, but also mouse wheel and page +/-, home and end keys).

I believe there could be valid use case for information about the following key types as well:

• compose, dead (accent) or hold (variants)
• escape/cancel/back or enter/return/submit/confirm/select/OK
• forward delete/erase and backwards delete/backspace
• (context) menu
• digits (numeric input)
  • row
  • phone-pad, T9
  • calculator-pad
• letters, syllabograms (linguistic input)
  • words (suggestions, autocomplete)
• colors (teletext)
• playback (like play/pause, skip, stop, record)