Interaction Media Features and Their Potential (for Incorrect Assumptions)

Celý článek

This is an updated and greatly expanded version of the article originally published on dev.opera back in 2015. That article referenced the Editor’s Draft, 24 March 2015 of the specification Media Queries Level 4, and contained a fairly big misunderstanding about how any-hover:none would end up being evaluated by browsers in practice.

The spec has since been updated (including clarifications and examples that I submitted following the publication of the original article), so this updated version removes the incorrect information of the original and brings the explanations in line with the most recent working draft. It also covers additional aspects relating to JavaScript touch/input detection.

The Media Queries Level 4 Interaction Media Features — pointer, hover, any-pointer and any-hover — are meant to allow sites to implement different styles and functionality (either CSS-specific interactivity like :hover, or JavaScript behaviors, when queried using window.matchMedia), depending on the particular characteristics of a user’s input devices.

Although the specification is still in working draft, interaction media features are generally well supported, though, to date, there are still some issues and inconsistencies in the various browser implementations — see the recent pointer/hover/any-pointer/any-hover test results, with references to relevant browser bugs.

Common use cases cited for interaction media features are often “make controls bigger/smaller depending on whether the users has a touchscreen device or is using a mouse/stylus” and “only use a CSS dropdown menu if the user has an input that allows hover-based interactions.”

@media (pointer: fine) {
  /* using a mouse or stylus - ok to use small buttons/controls */
}
@media (pointer: coarse) {
  /* using touch - make buttons and other "touch targets" bigger */
}
@media (hover: hover) {
  /* ok to use :hover-based menus */
}
@media (hover: none) {
  /* don't use :hover-based menus */
}

There are also examples of developers using these new interaction media features as a way of achieving standards-based “touch detection,” often just for listening to touch events when the device is identified as having a coarse pointer.

if (window.matchMedia && window.matchMedia("(pointer:coarse)").matches) {
  /* if the pointer is coarse, listen to touch events */
  target.addEventListener("touchstart", ...);
  // ...
} else {
  /* otherwise, listen to mouse and keyboard events */
  // ...
}

However, these approaches are slightly naive, and stem from a misunderstanding of what these interaction media queries are designed to tell us.

What’s the primary input?

One of the limitations of pointer and hover is that, by design, they only expose the characteristics of what a browser deems to be the primary pointer input. What the browser thinks, and what a user is actually using as their primary input, may differ — particularly now that the lines between devices, and the types of inputs they support, is becoming more and more blurry.

Right out of the gate, it’s worth noting that interaction media features only cover pointer inputs (mouse, stylus, touchscreen). They don’t provide any way of detecting if a user’s primary input is a keyboard or keyboard-like interface, such as a switch control. In theory, for a keyboard user, a browser could report pointer: none, signaling that the user’s primary input is not a pointer at all. However, in practice, no browser offers a way for users to specify that they are in fact keyboard users. So keep in mind that, regardless of what the interaction media feature queries may return, it’s worth making sure that your site or app also works for keyboard users.

Traditionally, we could say that a phone or tablet’s primary input is the touchscreen. However, even on these devices, a user may have an additional input, like a paired bluetooth mouse (a feature that has been available for years on Android, is now supported in iPadOS, and is sure land in iOS), that they are using as their primary input.

In this case, while the device nominally has pointer: coarse and hover: none, users may actually be using a fine pointer device that is capable of hovers. Similarly, if a user has a stylus (like the Apple Pencil), their primary input may still be reported as the touchscreen, but rather than pointer: coarse, they now have an input that can provide fine pointer accuracy.

In these particular scenarios, if all the site is doing is making buttons and controls bigger and avoiding hover-based interactions, that would not be a major problem for the user: despite using a fine and hover-capable mouse, or a fine but still not hover-capable stylus, they will get styling and functionality aimed at the coarse, non-hover-capable touchscreen.

If the site is using the cues from pointer: coarse for more drastic changes, such as then only listening to touch events, then that will be problematic for users — see the section about incorrect assumptions that can completely break the experience.

However, consider the opposite: a “regular” desktop or laptop with a touchscreen, like Microsoft’s Surface. In most cases, the primary input will be the trackpad/mouse — with pointer:fine and hover:hover — but the user may well be using the touchscreen, which has coarse pointer accuracy and does not have hover capability. If styling and functionality are then tailored specifically to rely on the characteristics of the trackpad/mouse, the user may find it problematic or impossible to use the coarse, non-hover-capable touchscreen.

For a similar take on this problem, see ”The Good & Bad of Level 4 Media Queries” by Stu Cox. While it refers to an even earlier iteration of the spec that only contained pointer and hover and a requirement for these features to report the least capable, rather than the primary, input device.

The problem with the original pointer and hover on their own is that they don’t account for multi-input scenarios, and they rely on the browser to be able to correctly pick a single primary input. That’s where any-pointer and any-hover come into play.

Testing the capabilities of all inputs

Instead of focusing purely on the primary pointer input, any-pointer and any-hover report the combined capabilities of all available pointer inputs.

In order to support multi-input scenarios, where different (pointer-based) inputs may have different characteristics, more than one of the values for any-pointer (and, theoretically, any-hover, but this aspect is useless as we’ll see later) can match, if different input devices have different characteristics< (compared to pointer and hover, which only ever refer to the capabilities of the primary pointer input). In current implementations, these media features generally evaluate as follows:

Going back to the original use cases for the interaction media features, instead of basing our decision to provide larger or smaller inputs or to enable hover-based functionality only on the characteristics of the primary pointer input, we can make that decision based on the characteristics of any available pointer inputs. Roughly translated, instead of saying “make all controls bigger if the primary input has pointer: coarse” or “only offer a CSS menu if the primary input has hover: hover,” we can build media queries that equate to saying, “if any of the pointer inputs is coarse, make the controls bigger” and “only offer a hover-based menu if at least one of the pointer inputs available to the user is hover-capable.”

@media (any-pointer: coarse) {
  /* at least one of the pointer inputs
    is coarse, best to make buttons and 
    other "touch targets" bigger (using 
    the query "defensively" to target 
    the least capable input) */
}
@media (any-hover: hover) {
  /* at least one of the inputs is 
     hover-capable, so it's at least 
     possible for users to trigger
     hover-based menus */
}

Due to the way that any-pointer and any-hover are currently defined (as “the union of capabilities of all pointing devices available to the user”), any-pointer: none will only ever evaluate to true if there are no pointer inputs available, and, more crucially, any-hover: none will only ever be true if none of the pointer inputs present are hover-capable. Particularly for the latter, it’s therefore not possible to use the any-hover: none query to determine if only one or more of the pointer inputs present is not hover-capable — we can only use this media feature query to determine whether or not all inputs are not hover-capable, which is something that can just as well be achieved by checking if any-hover: hover evaluates to false. This makes the any-hover: none query essentially redundant.

We could work around this by inferring that if any-pointer: coarse is true, it’s likely a touchscreen, and generally those inputs are not hover-capable, but conceptually, we’re making assumptions here, and the moment there’s a coarse pointer that is also hover-capable, that logic falls apart. (And for those doubting that we may ever see a touchscreen with hover, remember that some devices, like the Samsung Galaxy Note and Microsoft’s Surface, have a hover-capable stylus that is detected even when it’s not touching the digitizer/screen, so some form of “hovering touch” detection may not be out of the question in the future.)

Combining queries for more educated guesses

The information provided by any-pointer and any-hover can of course be combined with pointer and hover, as well as the browser’s determination of what the primary input is capable of, for some slightly more nuanced assessments.

@media (pointer: coarse) and (any-pointer: fine) {
  /* the primary input is a touchscreen, but
     there is also a fine input (a mouse or 
     perhaps stylus) present. Make the design
     touch-first, mouse/stylus users can
     still use this just fine (though it may 
     feel a big clunky for them?) */
}
@media (pointer: fine) and (any-pointer: coarse) {
  /* the primary input is a mouse/stylus,
     but there is also a touchscreen 
     present. May be safest to make 
     controls big, just in case users do 
     actually use the touchscreen? */
}
@media (hover: none) and (any-hover: hover) {
  /* the primary input can't hover, but
     the user has at least one other
     input available that would let them
     hover. Do you trust that the primary
     input is in fact what the user is 
     more likely to use, and omit hover-
     based interactions? Or treat hover 
     as as something optional — can be 
     used (e.g. to provide shortcuts) to 
     users that do use the mouse, but 
     don't rely on it? */
}

Dynamic changes

Per the specification, browsers should re-evaluate media queries in response to changes in the user environment. This means that pointer, hover, any-pointer, and any-hover interaction media features can change dynamically at any point. For instance, adding/removing a bluetooth mouse on a mobile/tablet device will trigger a change in any-pointer / any-hover. A more drastic example would be a Surface tablet, where adding/removing the device’s “type cover” (which includes a keyboard and trackpad) will result in changes to the primary input itself (going from pointer: fine / hover: hover when the cover is present, to pointer: coarse / hover: none when the Surface is in “tablet mode”).

If you’re modifying your site’s layout/functionality based on these media features, be aware that the site may suddenly change “under the user’s feet” whenever the inputs change — not just when the page/site is first loaded.

Media queries may not be enough — roll on scripting

The fundamental shortcoming of the interaction media features is that they won’t necessarily tell us anything about the input devices that are in use right now. For that, we may need to dig deeper into solutions, like What Input?, that keep track of the specific JavaScript events fired. But of course, those solutions can only give us information about the user’s input after they have already started interacting with the site — at which point it may be too late to make drastic changes to your layout or functionality.

Keep in mind that even these JavaScript-based approaches can just as easily lead to incorrect results. That’s especially true on mobile/tablet platforms, or in situations where assistive technologies are involved, where it is common to see “faked” events being generated. For instance, if we look over the series of events fired when activating a control on desktop using a keyboard and screen reader, we can see that fake mouse events are triggered. Assistive technologies do this because, historically, a lot of web content has been coded to work for mouse users, but not necessarily for keyboard users, making a simulation of those interactions necessary for some functionalities.

Similarly, when activating “Full Keyboard Support” in iOS’s Settings → Accessibility → Keyboard, it’s possible for users to navigate web content using an external bluetooth keyboard, just as they would on desktop. But if we look at the event sequence for mobile/tablet devices and paired keyboard/mouse, that situation produces pointer events, touch events, and fallback mouse events — the same sequence we’d get for a touchscreen interaction.

In all these situations, scripts like What Input? will — understandably, through no fault of its own — misidentify the current input type.

Incorrect assumptions that can completely break the experience

Having outlined the complexity of multi-input devices, it should be clear by now that approaches that only listen to specific types of events, like the form of “touch detection” we see commonly in use, quickly fall apart.

if (window.matchMedia && window.matchMedia("(pointer: coarse)").matches) {
  /* if the pointer is coarse, listen to touch events */
  target.addEventListener("touchstart", ...);
  // ...
} else {
  /* otherwise, listen to mouse and keyboard events */
  target.addEventListener("click", ...);
  // ...
}

In the case of a “touch” device with additional inputs — such as a mobile or tablet with an external mouse — this code will essentially prevent the user from being able to use anything other than their touchscreen. And on devices that are primarily mouse-driven but do have a secondary touchscreen interface — like a Microsoft Surface — the user will be unable to use their touchscreen.

Instead of thinking about this as “touch or mouse/keyboard,” realize that it’s often a case of “touch and mouse/keyboard.” If we only want to register touch events when there’s an actual touchscreen device for performance reasons, we can try detecting any-pointer: coarse. But we should also keep other regular event listeners for mouse and keyboard.

/* always, as a matter of course, listen to mouse and keyboard events */
target.addEventListener("click", ...);
 // ...

if (window.matchMedia && window.matchMedia("(any-pointer: coarse)").matches) {
  /* if there's a coarse pointer, *also* listen to touch events */
  target.addEventListener("touchstart", ...);
  // ...
}

Alternatively, we could avoid this entire conundrum about different types of events by using pointer events, which cover all types of pointer inputs in a single, unified event model, and are fairly well supported.

Give users an explicit choice

One potential solution for neatly circumventing our inability to make absolute determinations about which type of input the users are using may be to use the information provided by media queries and tools like What Input?, not to immediately switch between different layouts/functionalities — or worse, to only listen to particular types of events, and potentially locking out any additional input types — but to use them only as signals for when to provide users with an explicit way to switch modes.

For instance, see the way Microsoft Office lets you change between “Touch” and “Mouse” mode. On touch devices, this option is shown by default in the application’s toolbar, while on non-touch devices, it’s initially hidden (though it can be enabled, regardless of whether or not a touchscreen is present).

A site or web application could take the same approach, and even set the default based on what the primary input is — but still allow users to explicitly change modes. And, using an approach similar to What Input?, the site could detect the first appearance of a touch-based input, and alert/prompt the user if they want to switch to a touch-friendly mode.

Potential for incorrect assumptions — query responsibly

Using Media Queries Level 4 Interaction Media Features and adapting our sites based on the characteristics of the available primary or additional pointer input is a great idea — but beware false assumptions about what these media features actually say. As with similar feature detection methods, developers need to be aware of what exactly they’re trying to detect, the limitations of that particular detection, and most importantly, consider why they are doing it — in a similar way to the problem I outlined in my article on detecting touch.

pointer and hover tell us about the capabilities of whatever the browser determines to be the primary device input. any-pointer and any-hover tell you about the capabilities of all connected inputs, and combined with information about the primary pointer input, they allow us to make educated guesses about a user’s particular device/scenario. We can use these features to inform our layout, or the type of interaction/functionality we want to offer; but don’t discount the possibility that those assumptions may be incorrect. The media queries themselves are not necessarily flawed (though the fact that most browsers seem to still have quirks and bugs adds to the potential problems). It just depends on how they’re used.

With that, I want to conclude by offering suggestions to “defend” yourself from the pitfalls of input detections.

The post Interaction Media Features and Their Potential (for Incorrect Assumptions) appeared first on CSS-Tricks.

You can support CSS-Tricks by being an MVP Supporter.