Sushi Beats Drop

Over the past two months, our team has developed a rhythm game that combines traditional rhythm gameplay with innovative augmented reality (AR) elements. This fusion of classic mechanics with AR technology creates a unique and engaging experience that sets our game apart. Our primary goal was to design a game that is both fun and immersive, encouraging players to enjoy rhythmic challenges in an entirely new way.
This project builds on work we began during the last Niantic competition, where we explored creative applications of AR face filters. Our inspiration came from a sample project on Niantic Studio, which showcased head movement mechanics in AR. We were fascinated by how interactive and entertaining this concept could be, and we decided to make it a central mechanic in our game.

To determine the best gameplay style for this mechanic, we brainstormed ideas and drew inspiration from rhythm games like Taiko no Tatsujin and Osu. As fans of the genre, we envisioned a game that would incorporate not only timing and rhythm but also physical interaction through head movements. By merging these elements, we’ve created a game that stands out for its unique gameplay and physical engagement, offering players a rhythm game experience like no other.

Overview
Our rhythm game centers around a simple yet engaging concept: sushi drops from the top of the screen in sync with the music's rhythm. The player's goal is to catch the sushi at the right moment as it lands on the plates below. There are two plates positioned on the left and right
sides of the screen, and sushi pieces will drop toward each plate in rhythm with the beat of the music. When the sushi aligns with a plate, the player must take action to "catch" it, earning points for precise timing.

Controls and user inputs
Players can interact with the game using two different control methods, designed to provide flexibility and enhance the overall experience. The primary control method uses head tilts for a hands-free, immersive way to play. If sushi is falling toward the left plate, the player tilts their head to the left to catch it; the same applies to the right plate with a rightward tilt. This control style leverages augmented reality mechanics to add a layer of physical engagement and fun.
To accommodate players who might find head tilts challenging or uncomfortable over long sessions, we have also implemented a tap-to-play option. In this mode, players can tap the left or right side of the screen to catch sushi on the corresponding plate. This alternative ensures that the game remains enjoyable and accessible, catering to different preferences and levels of comfort.

By combining rhythm-based gameplay with intuitive and customizable controls, the game provides a fun and interactive experience that appeals to a broad audience.

3D Scene

Our game scene includes several essential components such as scripts, 3D assets, the face tracker, the camera, and lighting. While some of these, like the camera and lighting, are relatively intuitive, we will focus on the 3D assets and the face tracker in this section.

The face tracker plays a critical role in detecting the player's head position and movements. It allows the game to determine the direction and timing of head tilts, enabling accurate interaction with the gameplay mechanics.
The 3D assets include the two plates and the sushi pieces, both of which are essential to the rhythm game experience. The plates are static objects, each equipped with a physics collider and a static rigidbody component. A script, ScoreArea, is attached to each plate to handle scoring mechanics when sushi lands on them.

The sushi pieces, on the other hand, are dynamic objects. Each sushi has a physics collider and a dynamic rigidbody, allowing them to interact with the environment naturally as they fall. Additionally, each sushi piece is associated with a Sushi script that governs its behavior and integration into the gameplay.

The specific functionality of the scripts will be discussed in detail in a later section, but these components collectively create the interactive and immersive experience of our rhythm game.

Assets

All the assets in our game are entirely hand-drawn or custom-created, showcasing the artistic effort and attention to detail put into the project. The 3D models were crafted using Blender, adding a personal and unique touch to the game.
The assets include a variety of visual and interactive elements, such as the cover page, an instructional GIF to guide players, and animations for the gacha system. Additionally, we created a congratulation page for celebrating player achievements, as well as the two adorable cat sushi 3D models that serve as key gameplay elements.

Scripts

GameManager:

The gameManager handles the game's state transitions, including start, level selection, tutorial, in-game, rewards, and end screen. It dynamically manages UI elements, backgrounds, and event listeners for each state, ensuring smooth transitions and user interactions throughout the game.
The next two images show examples of different states from the state machine being defined.
Some key elements to notice are the events that are expected to transition to other events, as well as how each state creates and removes their own HTML UI from the page.

         
ecs.registerComponent({
  name: 'gameManager',
  stateMachine: ({ world, eid }) => {
    let startButton = null
    let levelButtons = []
    let endButton = null

    // Function to clean up the start button
    const removeStartButton = () => {}

    // Function to clean up level selection buttons
    const removeLevelButtons = () => {}

    // Function to clean up the end button
    const removeEndButton = () => {}

    ecs.defineState('startGame')
      .initial()
      .onEvent('interact', 'levelSelection', { target: world.events.globalId })
      .onEnter(() => {
        if (activeGameManagerEid !== null && activeGameManagerEid !== eid) {
          return
        }

        activeGameManagerEid = eid

        // Create the background and image
        createBackground(world)
      })
  }
})

         
ecs.defineState('levelSelection')
  .onEvent('levelSelected2', 'inGame', { target: world.events.globalId })
  .onEvent('showTutorial', 'tutorial', { target: world.events.globalId })
  .onEnter(() => {
    const levels = [
      { label: 'Slow', event: 'gameStartedSlow' },
      { label: 'Mid', event: 'gameStartedMid' },
      { label: 'Fast', event: 'gameStartedFast' },
    ]
  })

         
tick: (world, component) => {
  const { touchTimerLeft, touchTimerRight } = component.data
  const rotation = ecs.quaternion.get(world, component.eid)

  if (component.data.touchTriggeredLeft && world.time.elapsed > touchTimerLeft) {
    component.data.touchTriggeredLeft = false
  }

  if (component.data.touchTriggeredRight && world.time.elapsed > touchTimerRight) {
    component.data.touchTriggeredRight = false
  }

  if (rotation) {
    const z = rotation.z

    // Handle right-side logic
    if (z > 0.20) {
      component.data.hitLeft = false

      if (!component.data.hitRight) {
        component.data.hitRight = true
        component.data.canHitRight = true
      }

      if (component.data.hitRight && component.data.canHitRight) {
        handleRightSide(world, component)
      }
    } else if (z < -0.20) {
      // Handle left-side logic
      component.data.hitRight = false

      if (!component.data.hitLeft) {
        component.data.hitLeft = true
        component.data.canHitLeft = true
      }

      if (component.data.hitLeft && component.data.canHitLeft) {
        handleLeftSide(world, component)
      }
    } else {
      // Reset state when head returns to neutral
      resetHeadState(component)
    }
  }
}

         
// Dispatch global events on touch
world.events.addListener(world.events.globalId, ecs.input.SCREEN_TOUCH_START, (event) => {
  const touchX = event.data?.position?.x
  if (touchX < 0.5) {
    world.events.dispatch(eid, 'touchLeft')
  } else {
    world.events.dispatch(world.events.globalId, 'touchRight')
  }
})

// Listen for global touch events within this component
world.events.addListener(eid, 'touchLeft', () => {
  const data = dataAttribute.cursor(eid)
  data.touchTriggeredLeft = true
  data.touchTimerLeft = world.time.elapsed + 1000
})

world.events.addListener(world.events.globalId, 'touchRight', () => {
  const data = dataAttribute.cursor(eid)
  data.touchTriggeredRight = true
  data.touchTimerRight = world.time.elapsed + 1000
})

ObjectSpawner:

The objectSpawner component handles the spawning of sushi objects at predefined timestamps, synchronized with audio tracks of varying speeds. It randomizes spawn locations and sushi types, sets up event listeners for different game modes, and manages audio playback to deliver a dynamic gameplay experience driven by global
events.
In order to spawn the sushi we have references to both types in the scene which already have the appropriate components. We use the following code to duplicate them and then have then fall down.
We create a new entity, the target entity, and then copy all of the components from the source into it.

         
function spawnSushi(world, objectToSpawn, objectToSpawnSuper, spawnY, spawnZ, timeStamps) {
  if (currentTimestampIndex >= timeStamps.length)
    return

  const sushiType = randomizeSushi()
  const newEid = world.createEntity()
  const spawnX = randomizeSpawnLocation()

  const clonedSuccessfully = sushiType === "regular"
    ? cloneComponents(objectToSpawn, newEid, world)
    : cloneComponents(objectToSpawnSuper, newEid, world)

  if (!clonedSuccessfully) {
    world.deleteEntity(newEid)
    return
  }
}

// Clone components from the source to the target entity
const cloneComponents = (sourceEid, targetEid, world) => {
  const componentsToClone = [
    Position, Quaternion, Scale, Shadow, BoxGeometry, Material,
    ecs.PositionAnimation, ecs.RotateAnimation, ecs.GltfModel,
    ecs.Collider, ecs.Audio, Sushi
  ]

  let clonedAnyComponent = false

  componentsToClone.forEach((component) => {
    if (component && component.has(world, sourceEid)) {
      const properties = component.get(world, sourceEid)
      component.set(world, targetEid, { ...properties })
      clonedAnyComponent = true
    }
  })

  return clonedAnyComponent
}

         
currentTimestampIndex++

if (currentTimestampIndex < timeStamps.length) {
  const delay = (timeStamps[currentTimestampIndex] - timeStamps[currentTimestampIndex - 1]) * 1000
  setTimeout(() => spawnSushi(world, objectToSpawn, objectToSpawnSuper, spawnY, spawnZ, timeStamps), delay)
}

         
// Slow - One
const timeStampsSlow = [
  0.22, 1.31, 2.5, 3.8, 5.08, 6.37, 7.66, 8.95, 10.23, 11.53, 12.76, 14.1, 15.39,
  16.68, 17.97, 19.26, 20.55, 21.85, 23.14, 24.43, 25.72, 27.01, 28.3, 29.59,
  30.88, 32.16, 33.45, 34.74, 36.03, 37.32, 38.61, 39.9, 41.19, 42.49, 43.79,
  45.07, 46.36, 47.65, 48.94, 50.23, 51.52, 52.79, 54.1, 55.39, 56.68, 57.97,
  59.26, 60.55, 61.85, 63.14, 64.43, 65.72, 67.01, 68.3, 69.59, 70.88, 72.17,
  73.66, 75.01, 76.18, 77.59, 78.7, 79.93, 81.19, 82.49, 83.78, 85.07, 86.36,
  87.65, 88.94, 90.23, 91.52, 92.81, 94.1, 95.39, 96.68, 97.97, 99.26, 100.55,
  101.83, 103.14, 104.43, 105.72, 107.01, 108.3, 109.59, 110.88, 112.17, 115.52
]

         
const handleCollisionStart = (e) => {
  if (Sushi.has(world, e.data.other)) {
    const areadata = component.schemaAttribute.get(eid)
    areadata.hasSushi = true
    areadata.sushiEntity = e.data.other

    ecs.Material.set(world, eid, { r: 225, g: 225, b: 0 })
    const rightScoreAreaData = Sushi.get(world, areadata.sushiEntity)

    if (rightScoreAreaData.type === "regular") {
      areadata.score = Math.floor(Math.random() * 3) + 1
    } else if (rightScoreAreaData.type === "super") {
      areadata.score = 10
    } else {
      areadata.score = -1
    }
  }
}

         
if (ecs.Position.has(world, eid)) {
  const currentPosition = ecs.Position.get(world, eid)

  if (currentPosition) {
    currentPosition.y -= sushiData.speed

    if (currentPosition.y < -5.0) {
      // console.log(`Sushi (${eid}) went off-screen, deleting entity.`)
      world.deleteEntity(eid)
    } else {
      ecs.Position.set(world, eid, currentPosition)
    }
  }
}

UIRewardController:
● The UIRewardController component manages the reward display system, animating and
showcasing rewards such as S, SS, and SSS with corresponding GIFs and points. It
dynamically updates the UI, handles event-based reward additions, and triggers the
reward sequence, ensuring an engaging post-game experience.

Rewards:
● The Rewards component manages the reward system, displaying popups for events like
Combo3 and triggering rewards for Combo10, including a Mega Bonus Gacha with
randomized rewards (S, SS, or SSS). It tracks reward data, listens for global events, and
provides engaging feedback to players through visual and event-driven rewards.

SushiKiller:
● The sushiKiller component detects collisions with sushi entities and dispatches a global
comboReset event upon collision. It manages state data for tracking sushi presence and
collection status, ensuring game mechanics are updated based on interactions.

❖ Start Game
Transitions To: Level Selection
Description: The initial screen of the game where players begin their journey.
UI Elements: A background and a button to transition to the Level Selection
screen.

❖ Level Selection
Transitions To: Tutorial, Main Gameplay
Description: Players can choose from three songs, each associated with a
specific difficulty level.
UI Elements: A background, three buttons representing the available songs, and
a single button to transition to the Tutorial state.

❖ Tutorial
Transitions To: Level Selection

Description: This state teaches players how to interact with the game through
two mechanics:
■ Moving their head left to right to capture falling sushi on a plate.
■ Tapping the screen when falling sushi touches the plate.
Animated GIFs demonstrate these actions, and a button allows players to
return to the Level Selection screen.

➢ UI Elements: Animated GIFs showing gameplay mechanics and a button to
transition back to Level Selection.
➢ Main Gameplay
Transitions To: Reward Screen
Description: The core gameplay section where players interact with the
game:
● Players see themselves on screen as sushi falls to the rhythm of
the music.
● The goal is to collect sushi to increase their score and combo
counter.
● Higher combos grant better rewards in the Gacha system.
UI Elements: Live player feed, falling sushi visuals, score counter, and
combo tracker.
➢ Reward Screen
Transitions To: End Game Screen
Description: After completing a song, players receive rewards based on their
performance:
● The screen displays animations of the Gachas they earned.
● Players see their final score and any score increases from the
session.
● Once the animations finish, the game transitions to the End Game
Screen.

UI Elements: Reward animations, score summary, and transition trigger to
the End Game Screen.

➢ End Game Screen
Transitions To: Level Selection
Description: The final screen of the game session, thanking players for
playing.
● Players are prompted to start another round by returning to the
Level Selection screen.

UI Elements: A thank-you message and a button to return to the Level
Selection screen.

Sushi Beats Drop

Make it your own with the sample project

Head Touch Input

Object Cloning and Detection

Behind the Build: Sushi Beat Drop

Summary

Gameplay

Project Structure

3D Scene

Assets

Scripts

Implementation