Video Playback System

The video playback system is designed to show cinematic FMV scenes that are fully integrated into gameplay, rather than separate pre-rendered cutscenes.

Instead of playing videos in a fixed way, the system can react to what is happening in the game. This allows scenes to change dialogue, trigger events, or adjust outcomes while the video is still playing, making storytelling more dynamic and connected to gameplay.

The system plays MPEG-1 and MPEG-2 video files using a dedicated decoding pipeline.

FMV audio is played as streamed audio, using formats such as Dolby AC-3, CRI ADX, and CRI AHX depending on the scene. This is used for background music and cinematic sound during playback.

Dialogue is handled separately as its own mono track, so it can be controlled independently from the background audio. This improves timing accuracy, language support, and flexibility during playback.

Each FMV scene is made up of multiple synchronized parts: video, background audio, dialogue, and subtitle text tracks. Subtitles are fully timed and can be changed during playback for different languages or story situations.

The system also includes a built-in control layer (virtual machine) that runs instructions during playback. It handles timing, events, subtitles, and communication with the game, allowing FMV scenes to respond to gameplay in real time.

The video system supports alpha FX masking, where a grayscale mask defines transparency on MPEG-1 and MPEG-2 video. This allows FMV elements to blend into the 3D world.

FMV output can also be used as a texture on 3D objects or 2D screen, allowing video playback to appear on surfaces within the game world instead of only on screen overlays.

Overall, the system combines video, audio, dialogue, subtitles, visual effects, and game logic so FMV scenes feel like a natural part of the game world.

On the Macronix 16-bit Audio DSP, video playback system is designed specifically around the hardware to keep FMV scenes smooth and properly synchronized.

Audio is handled by the system’s dedicated audio DSP. It runs its own microcode, which is a small set of instructions that tells it how to process audio in real time. During FMV playback, the DSP takes care of playing and mixing the streamed audio independently from the main processor.

Because the audio DSP is doing its own work, the main CPU is free to focus on video playback, cinematic logic, and subtitle timing. This separation helps keep everything running smoothly without slowing the game down.

The result is that video, audio, and subtitles all stay tightly in sync, while each part of the system handles what it is best at: the CPU handles visuals and logic, and the audio DSP handles sound processing.