I Tried an AI Cover, Saw a 15-Hour Estimate, and Quit

It started with an AI Sun Yanzi cover.

It was a Jay Chou song, but the voice coming out sounded uncannily like Sun Yanzi, right down to the breath and diction. My mental model was wonderfully simple: separate vocals from the backing track, turn the vocals into someone else's voice, done.

So I picked a target just as quickly: JJ Lin singing “Zhi Wo”, the theme song of Jian Lai.

Then one very basic question arrived: where was JJ Lin's voice supposed to come from?

What is it actually changing?

It was less mysterious than I expected, but it was not a “JJ Lin filter” either. RVC does not sing the song again. It acts more like a voice converter: take an existing vocal performance, preserve its melody, timing, and most of its breathing, then replace the part that answers “who is singing?”

Take “Zhi Wo.” The original singer's melody, timing, and roughly the same breaths stay in place. The model's job is to change who appears to be singing: the original voice becomes the target voice. Only after that do you mix the processed vocal back with the instrumental.

That is why the song first has to be split into vocals and instrumental. Feed in the full mix and the drums, bass, and reverb all get dragged through conversion too, which is a good recipe for metallic soup. A dry vocal is simply the cleanest voice-only stem you can make.

I found the songs and the material, but not the model

The original and instrumental versions of “Zhi Wo” were both on Bilibili, and I found lossless copies. For training material I picked Practice Love, The Key, Exchange of Lives, and She Says: four lossless tracks, 18 minutes in total.

At this point I was still optimistic. Finding, separating, and organising the audio took about two hours, and the imaginary progress bar in my head looked half complete. It turned out to be mostly decorative.

I searched GitHub and Hugging Face, then model sites, then a few Bilibili collections claiming to contain hundreds of voice packs. The result was clean: zero JJ Lin models in the public sources I could find.

At first I assumed I was searching badly. After trying a few places, it became clear that the problem was not the keyword. Voice models of real singers have mostly been scrubbed, just like the Sun Yanzi ones. The model file itself is easy enough to understand; the hard part is getting one that you can actually use.

Training my own was the only path left.

When I read that RVC could learn a voice from a few songs, I was briefly delighted. Really? Four tracks go in and a model comes out? I chose local training. Clicking that option felt light; the “estimated fifteen hours” that came later did not.

Most “make an AI cover in minutes” tutorials quietly assume the voice model already exists. That was exactly the piece I did not have, so I had to start with training.

Fifteen hours

I had compressed “training a model” and “using a model” into one step. Training is the heavy job: the machine learns a voice from a batch of clean vocals. Applying an existing model to one song is much lighter. Videos with a result in minutes tend to leave the slow upper half outside the frame.

I didn't have an existing model, so I had to start with the heavy part.

RVC training prefers a GPU. I did not have one, so an Apple M5 CPU took the shift. A normally quiet computer suddenly made sure I knew it was there.

Each epoch took five to six minutes, with 150 epochs configured. I opened a calculator: fifteen hours.

I assumed I had calculated it wrong. I tried again. Still fifteen hours. Local training had not lied to me; it was simply being very honest.

I gave up looking at the progress bar

I waited half an hour and finished only a handful of epochs. The pipeline worked, the data was valid, and the logs looked perfectly healthy. The only problem was that they intended to stay healthy until tomorrow.

The fifteen-hour estimate told me I had chosen the wrong route. I did not want this cover badly enough to put the computer on a night shift. I killed the process and deleted the Python environment, RVC, training data, and separated audio. More than eight gigabytes, gone.

The funny part: the fan kept spinning even after the deletion. Turns out macOS was re-indexing in the background, since I had just deleted eight gigabytes of files. Even giving up came with interest.

Looking back, RVC was capable and the material was fine. I had simply reached the “local or cloud” fork and picked the branch with the most entertainment value.

I looked into the cloud option afterward. Entry-level GPUs on AutoDL are rented by the hour and cost only a few yuan. The exact duration depends on the card and settings, but at least the question changes from “should my computer run all night?” to “am I willing to spend a few yuan?” The source material can be prepared again. Next time, I will go straight to the cloud.

Real people's voices also come with consent and copyright questions. Not finding a JJ Lin model was not entirely bad; at least I did not casually publish something made with a file of unknown origin. If I return to this, I will start with my own voice.

There is no finished cover, and I never reached inference. I did understand the workflow. Next time, rent the GPU first.