VALL-E Demo

This demo page contains samples from my VALL-E implementation: https://git.ecker.tech/mrq/vall-e/.
Unlike the original VALL-E demo page, I'm placing emphasis on the input prompt, as the model adheres to it stronger than others.
Samples are provided as-is without trying to gamble for "the best", as I aim to show possible quirks that may occur.
Samples are likely to update over time as I'm nailing out sampling issues and performing additional training on the model.

LibriSpeech

Below are samples from the LibriSpeech corpus to demonstrate common speech from possibly unseen speakers.
Samples are compared against the supplied samples from the original VALL-E's demo page, as well as outputs from F5-TTS.

Text Prompt Our VALL-E Original VALL-E F5 Ground Truth
Number ten, fresh nelly is waiting on you, good night husband.
Yea, his honourable worship is within, but he hath a godly minister or two with him, and likewise a leech.
Instead of shoes, the old man wore boots with turnover tops, and his blue coat had wide cuffs of gold braid.
The army found the people in poverty and left them in comparative wealth.
Thus did this humane and right minded father comfort his unhappy daughter, and her mother embracing her again, did all she could to soothe her feelings.
He was in deep converse with the clerk and entered the hall holding him by the arm.
They moved thereafter cautiously about the hut groping before and about them to find something to show that Warrenton had fulfilled his mission.
And lay me down in thy cold bed and leave my shining lot.

Sampled Dataset

Below are samples from the dataset used to train the model.
These samples should cover both non-English speech (through the Emilia dataset) and speakers not typically considered for TTS (for example: sourced from videogames).
Samples from F5-TTS are also provided as-is (default settings).

Text Prompt Our VALL-E F5 Ground Truth
Das sind so die Sachen, die ich persönlich euch da ans Herz legen kann.
Transport public du Chablais. Il exploite 69 km de voies métriques autour d'Aigle.
I do not know why but I always want to be by your side.
Unlike the other buildings, this one's co-ed. It's not your typical dorm though.
Should you opt to accept my request, bring this to me, where I will wait.
You've had enough, right? Please, show some mercy!
I'll tell you what that thing isn't. It isn't yours, so leave it alone.
やれやれ。連長車をコキ使うもんじゃねえぞ。
Sampled Dataset
Text Prompt Our VALL-E Ground Truth
Das ist übrigens Vergangenheit, also nicht denken, es bricht jetzt ein Vulkan aus.
What? Are you saying you forgive this bastard?
We're dealing with a murderer here, but we've all risked our lives to get this far.
Yeah, I'm thinking about going in again. I want to find out for myself.
Say, did you hear about the incident from yesterday? They found that announcer lady hanging from an antenna.
It's just this guy's personal rankings, nothing important. What does the next one say?
How dare she tear away our innocence like this! Now I'm scarred for life, dammit!

The remaining sections cover how speakers performed when using a LoRA.
The rows with "Ground Truth" reference audio are sampled from the dataset the LoRA was trained against, while rows without sampled input texts from the validation dataset (to show how the speaker performs beyond what was trained against).

Portal: GLaDOS

Text Prompt Our VALL-E (LoRA) Our VALL-E (No LoRA) Ground Truth
Let's be honest. Neither one of us knows what that thing does. Just put it in the corner and I'll deal with it later.
Once again, the Enrichment Center offers its most sincere apologies on the occasion of this unsolvable test environment.
However, it cannot accompany you for the rest of the test and, unfortunately, must be euthanized.
Despite the best efforts of the Enrichment Center staff to ensure the safe performance of all authorized activities, you have managed to ensnare yourself permanently inside this room.
He was looking downcast and sad, and it was easy to see he was disappointed because he had not conquered the bularu himself.
Alexander, annoyed by Mainhall's sighs and exclamations, watched her with keen, half-skeptical interest.
The dews were suffered to exhale, and the sun had dispersed the mists, and was shedding a strong and clear light in the forest when the travelers resumed their journey.
It could not be used for electroplating or deposition, nor could it charge storage batteries, all of which are easily within the ability of the direct current.

Sam & Max: Sam

Text Prompt Our VALL-E (LoRA) Our VALL-E (No LoRA) Ground Truth
Look at this. It's Santa's magical stocking stuffer. It fills a sock with whatever gift the owner deserves.
What? Oh, no, Commissioner, I didn't mean... Look, I need you to put out an APB on Hugh Bliss. Stat!
Nomex, we have a far more bloodthirsty adversary this time. The President of the United States of America.
It says here that two members of the United States Congress are secretly involved with a silicon-based rock beast from Venus.
Many of these affiliated gentlemen of leisure are at the same time less a man of substance in their own right, so that some of them are scarcely at all, others only partially, to be rated as vicarious consumers�
I see, replied Socrates, that I shall be well entertained, and do you, Timaeus, offer up a prayer and begin.
Here as elsewhere, prescription seizes upon the custom as soon as it gains a vogue and incorporates it in the accredited standard of decency.
I shall soon be as well as I am ever likely to be. That is quite well," answered Rachel.

The following speakers are unseen to the model, so their non-LoRA outputs are a bit lacking.
It needs to be noted that the samples the LoRA was trained on were rather sloppily transcribed and sliced, as they're sourced from a YouTube video without proper segmentation.
Common issues include samples truncated too early and long pauses between utterances.

Cyberpunk 2077: Johnny Silverhand

Text Prompt Our VALL-E (LoRA) Our VALL-E (No LoRA) Ground Truth
Haha, seriously? What, you think you're gonna find a jack-in-the-box who wants to become a real boy?
Rogues on board, too. Told me to tell you hi. Pretty funny, right? Like old times. Alt.
Let's take a look, shall we? What we came here to do. Find the truth. Ask her.
Sorry if I'm unable to hand you all the answers on a chrome fucking platter when you snap your fingers.
her slate blue skirts were kilted boldly about her waist and dovetailed behind her.
To teach reading means to light the fire.
But suddenly, straight and swift as a diving cormorant, he shot down into the torrent and and disappeared beneath the surface.
yet the deep marks of conical teeth upon the iron pick are certainly those of a crocodile.

Cyberpunk 2077: Delamain

Text Prompt Our VALL-E (LoRA) Our VALL-E (No LoRA) Ground Truth
Do you hear me? Junior, as it were. I'm afraid I can't help you with the former,
premium clientele. Unfortunately, we do not take on such contracts.
Good morning V. Absolutely. Yes, of course. We'll be taking those special air corridors.
That which was divided has become one. It's time to go home.
A great white foam was spread upon its brown and green. The whole field was waving and shivering in the sunlight.
near the songster, in the dark background of the large trees, could be seen the glistening eyes of an owl, attracted by the harmony.
At any rate, my eloquence was altogether stopped. The gentleman was named Sir Ferdinando Brown.
Little by little, however, the latter became hemmed and bound in the meshes of the various devices and proceedings which the territorial officials evolved from the bogus laws.

Settings used:

{'input_prompt_length': 0.0, 'max_ar_steps': 900, 'max_nar_levels': 7, 'ar_temp': 0.0, 'nar_temp': 0.0, 'min_ar_temp': -1.0, 'min_nar_temp': -1.0, 'top_p': 1.0, 'top_k': 0, 'min_p': 0.0, 'repetition_penalty': 1.0, 'repetition_penalty_decay': 0.0, 'length_penalty': 0.0, 'beam_width': 0, 'mirostat_tau': 0, 'mirostat_eta': 0, 'dry_multiplier': 0, 'dry_base': 1.75, 'dry_allowed_length': 2, 'entropix_sampling': False}