Building Diphone
database
Although diphone-based
synthesizers are only one of a number of techniques
documented in FestVox, we believe they are, at present,
the most reliable and resource-effective method for
building new voices for general text-to-speech synthesizers.
A diphone here is two connected half-phones, where a
"phone" here may in fact be any segment including
a traditional phoneme, allophone or consonant cluster.
We carefully construct examples of each phone-phone
transition in our phoneset[appendix 2], so as to capture
all the implied sequential articulatory transitions,
even though some may not be phonotactically valid (like
[n:-ae]).
To fully exercise our techniques, we are collecting
one sets of diphones, at 32KHz,from a single speaker
of Bangla (IAR) with varying speaking rate. These databases
are being released publicly with an open license, so
that anyone who wishes can replicate our findings, study
the voice, teach about synthesis, or build their own
by comparison.
Diphone Synthesis
Diphone synthesis is one of the most popular methods
used for creating a synthetic voice from recordings
or samples of a particular person; it can capture a
good deal of the acoustic quality of an individual,
within some limits. The rationale for using a diphone,
which is two adjacent half-phones, is that the ``center''
of a phonetic realization is the most stable region,
whereas the transition from one ``segment'' to another
contains the most interesting phenomena, and thus the
hardest to model. The diphone, then, cuts the units
at the points of relative stability, rather than at
the volatile phone-phone transition, where so-called
coarticulatory effects appear.
There is clearly a simplifying assumption: that all
relevant phonetic realizations can be enumerated, and
that by simply collecting all of phone-phone transitions,
that any possible sequence of speech sounds in the target
language could be produced. Thus, with a 44-phone inventory,
one could collect a 44 * 44 = 1936 diphone inventory
and create a synthesizer that could speak anything,
given the imposition of appropriate prosody - intonation,
duration, and shift in spectral quality, as determined
by other modules in a general-purpose synthesizer.
For Bangla we had 2108 nonsense words generated by our
script and some of the non sense words contain invalid
combination so we manually excluded them. Some changes
in the Bd_schema.scm can result in better diphone list.
After hand correction we had 1410 nonsense words which
contained 2820 diphones for bangla. Using this diphone
list any speech can be produced including consonant
clusters.
Collecting a Diphone Set
Building a diphone synthesizer involves several steps:
• Choosing the phone set
• Designing carrier material
• Generating the prompts
• Preparing the audio collection environment
• Collecting the data
• Segmentation/Alignment
• Quality control
• Extracting pitchmarks and pitch-synchronous parameters
• Diphone index
• Amalgamation
Choosing the phone set
One should consider the phone set carefully for synthesis.
For Bangla there were no phoneset defined so we had
to make for our own purpose. Many may not agree with
the phoneset but it served us well for our diphone generation
and for other modules. The main objective of building
a phoneset for a language is to define all the phonemes
and other sound for the festival system.
Designing carrier material
We use nonsense carrier words to collect all possible
diphones, following [5]. Others have successfully used
natural carrier phrases, but the argued advantage of
natural delivery offers may also be a disadvantage as
people may assume too much, and fail to produce exactly
the desired phones. Within this framework, an experiment
may be carried out to compare the results of voices
made from nonsense words to one made from naturalistic
text, but we know of none having been performed as of
yet.
It should be noted that delivering diphones is not a
particularly natural endeavor. As these phone segments
will be extracted both for pitch and duration, it is
important that their delivery be consistent, so that
joins are more likely to be acceptable.
We believe that the use of nonsense carrier material
makes the delivery of the diphones more consistent.
Also, the pronunciation of a phonetic string is more
clearly defined in these nonsense words than in elicited
natural words. We generate carrier phrases so that,
where possible, we can extract the diphones from the
middle of a word. As it takes time for the human articulation
system to start, we do not want to extract diphones
from syllables at the start or end of words, unless
these transitions to or into silence (SIL) are part
of the diphone in question.
for example, from
# t aa k a k aa #
we would extract the diphones [k-a] and [a-k], and from
# t aa kh a kh aa #
we would get [kh-a] and [a-kh] from the prompt.
For each class in the language, consonant-vowel, vowel-consonant,
vowel-vowel, consonant-consonant, silence-phone, phone-silence,
and any other special sets like syllabic, consonant
clusters and allophones we build simple carrier sets
and loop through all possible values generating a long
list of strings of phones which contain all possible
diphones in the language to be recorded. Basic scripts
are provide for this that can be adapted for other language,
while specific scripts are provided for currently supported
languages.
Generating the
prompts
Once the phone set and the sort of carrier material
for each type has been chosen, the prompt list is produced
automatically from the specification using a set of
templates. Consonant-vowel and vowel-consonant pairs
are generally kept together, as shown in the example
above - [k-a] and [a-k] are generated in a single prompt.
Special contexts are also created for e.g. vowel-vowel
diphones, and for transitions to and from silences into
a phone.
We then synthesize these prompts using an existing synthesizer.
The prompts are generated for a number of reasons: first,
to play to the user while recording. Even highly trained
phoneticians make mistakes in reading 2000 prompts so
the synthesized prompts help guide the speaker to say
the right thing. Secondly, as we generate these prompts
at constant pitch and duration, this encourages the
speaker to do likewise. As we are going to modify the
pitch and duration independently, it is better if the
recording is in a monotone, and consistent. This, we
feel, is best done by effectively having the nonsense
word delivered in what almost sounds like a chant.
The second reason for generating prompts is their use
in labelling the spoken word which is described below.
Preparing the
audio collection environment
We collected the diphones at home and try to make it
as much clear and noiseless. For collecting voice data
we used several software. We collected the diphones
at Windows Xp environment as the sound editing facility
is better than Linux there. For sound collection the
default XP sound recorder can be used but for better
quality we used Sourge Forge. With Sourge Forge the
effect of the environment can be minimized so that the
noise is reduced. With Sourge Forge we had to do the
following –
1. Monitor Input level
2. Choose an input device and adjust levels for that.
3. Adjust for DC offset
4. Preparing for recording
5. Review recorded intakes.
Fig. Adjusting audio recording input level.
After the recording of the nonsense words its worth
to make the voice levels of all files at the same level
as voice level may differ from file to file. This helps
to keep the consistency of the voice quality of the
synthesizer otherwise there will be a level factor error
at the synthesized speech output. For leveling the audio
signals we used the Wave Hammer tool of Sourge Forge.
We choose “Level at 6 db and maximize” option to level
our audio files.
Collecting the data
For collecting data
we used the sourge forge editor. Using the magnifying
tool the diphone segments can be extracted perfectly
so that no silence part are included in the segments.
Otherwise that could rduce te quality of diphone database.
Fig. Wave hammer for
leveling audio signals.
next>>
|