# Model: Hands

Overview and basic demo
  • πŸ– 21 2D hand landmarks
  • πŸ–πŸ– Track up to 4 hands total

This model includes dozens of Pinch Events and helper styles to get you going quickly, along with a plugin for scrolling pages handsfree.

# Usage

# With defaults

const handsfree = new Handsfree({hands: true})
handsfree.start()

# With config

const handsfree = new Handsfree({
  hands: {
    enabled: true,
    // The maximum number of hands to detect [0 - 4]
    maxNumHands: 2,

    // Minimum confidence [0 - 1] for a hand to be considered detected
    minDetectionConfidence: 0.5,

    // Minimum confidence [0 - 1] for the landmark tracker to be considered detected
    // Higher values are more robust at the expense of higher latency
    minTrackingConfidence: 0.5
  }
})

handsfree.start()

# Data


Image source, MediaPipe: https://google.github.io/mediapipe/solutions/hands#hand-landmark-model (opens new window)

# Hand Landmarks

// handIndex [0 - 3] An array of landmark points for each detected hands
handsfree.data.hands.multiHandLandmarks[handIndex] == [
  // Landmark 0
  {x, y},
  // Landmark 1
  {x, y},
  // ...
  // Landmark 20
  {x, y}
]

// hand 0, landmark 0
handsfree.data.hands.multiHandLandmarks[0][0].x
handsfree.data.hands.multiHandLandmarks[0][0].y

# Is it the right or left hand?

// handIndex [0 - 3] An array of landmark points for each detected hands
handsfree.data.hands.multiHandedness[handIndex] == {
  // "right" or "left"
  label,
  // The probability that it is "right" or "left"
  score
}

// hand 0
handsfree.data.hands.multiHandedness[0].label
handsfree.data.hands.multiHandedness[0].score

# Examples of accessing the data

handsfree = new Handsfree({hands: true})
handsfree.start()

// From anywhere
handsfree.data.hands.multiHandLandmarks

// From inside a plugin
handsfree.use('logger', data => {
  if (!data.hands) return

  console.log(data.hands.multiHandLandmarks[0])
})

// From an event
document.addEventListener('handsfree-data', event => {
  const data = event.detail
  if (!data.hands) return

  console.log(data.hands.multiHandedness)
})

# Examples

Scroll pages handsfree
Person scrolling page by pinching the air and moving hand up and down to scroll in that direction

The pinchScroll plugin helps you scroll pages with a pinch gesture. In the GIF above, it's being used within a Browser Extension to scroll pages you visit handsfree.

Add your project
If you've made something with this model I'd love to showcase it here! DM me on Twitter, make a pull request, or find us on Discord.

# See Also

  • palmPointers plugin - Creates pointers for each hand that can be moved around my moving the hands with the palm pointed towards the webcam
  • pinchers plugin - This plugin adds dozens of new events and helper styles for pinching any finger (index, middle, ring, pinky) to your thumb. It is enabled by default
  • pinchScroll plugin - Adds the ability to scroll pages with a pinch gesture
Debugger