Skip to main content

MuseTalk Provider Guide

Lip-synced avatar videos via the MuseTalk model on Replicate

Overview​

MuseTalk is a low-latency open-source lip-sync model hosted on Replicate. NeuroLink wraps it under output.avatar.provider = "musetalk".

Key Facts​

  • Hosting: Replicate Predictions API
  • Model: e.g. cjwbw/musetalk:...
  • Async: Submit + poll
  • Output: MP4

Quick Start​

1. Get a Replicate Token​

https://replicate.com/account/api-tokens

2. Configure​

REPLICATE_API_TOKEN=r8_your-token

3. Generate​

import { NeuroLink } from "@juspay/neurolink";
import { readFileSync, writeFileSync } from "node:fs";

const ai = new NeuroLink();
const result = await ai.generate({
  provider: "vertex",
  output: {
    mode: "avatar",
    avatar: {
      provider: "musetalk", // routes to Replicate-hosted MuseTalk
      image: readFileSync("./portrait.jpg"),
      audio: readFileSync("./narration.mp3"),
    },
  },
});
if (result.avatar?.buffer) {
  writeFileSync("./avatar.mp4", result.avatar.buffer);
}

CLI Usage​

pnpm run cli generate "" \
  --provider musetalk \
  --avatarImage ./portrait.jpg \
  --avatarAudio ./narration.mp3 \
  --avatarOutput ./avatar.mp4

Configuration Reference​

Environment VariableRequiredDescription
REPLICATE_API_TOKENYesReplicate token (shared)

See Also​