Diphone

Phones are not a suitable unit for waveform concatenation, so we used diphones, which capture co-articulation.

Diphone starts at the middle of one phone and ends at the middle of the other.

Coarticulation is the overlapping of adjacent articulations or the influence of the target phoneme on surrounding phonemes. Middles of phones are more stable in their spectral properties than the edges, because of coarticulation. So, concatenating diphones should lead to smoother joins

s18221310312022

s22373310222022

Waveform concatenation

Concatenation of waveforms is a simple way of making synthetic speech, but we need to take care about how we do it.

  1. discontinuity cause pops
  2. periodicity alignment cause glitches

s22424210222022

s22432010222022

s18440910312022

Overlap-add

Cross-fading between two waveforms is an effective way to avoid some of the artefacts of concatenation.

s22460610222022

s22461610222022

Pitch period

This fundamental building block of speech waveforms offers a route to source-filter separation in the time domain.

Overlap of pitch period or impulse signal is observed.

s22524010222022

extract pitch period (with taper window) for each pitch mark, and we make the time for each pitch period twice the $T_0$.

s22571610222022

overlap to get the reconstruction signal similar to the original one. the whole process is called copy sentences.

s22573710222022

s22575410222022

TD-PSOLA

Applying overlap-add techniques to pitch period waveforms allows the modification of F0 and duration without changing the phone identity.

Time-domain pitch-synchronous overlap-and-add

Pitch period closer to each other
s23105710222022

Pitch period far apart from each other
s23112110222022

make a copy of one pitch period and insert to the sequence.
s23120110222022

delete one pitch period
s23121510222022

Diphone synthesis:

  • One recording of every diphone (small database)
  • Use signal processing methods to change F0, duration, and smooth joins to match linguistic specification
  • e.g. TD-PSOLA

Unit Selection

Unit selection:

  • Record a large naturalistic database
  • Select diphone units based on closeness to the linguistic specification
  • If the database has enough variation, don’t worry (too much) about signal processing!

Choice of units to concatenate depends on:

  • Target cost: how well the unit matches the linguistic specification
  • Join cost: how well edges of the units match

Convolution

A non-mathematical illustration of the equivalence of convolution (in the time domain), multiplication of magnitude spectra, and addition of log magnitude spectra.

s23205410222022

Summary

s23225210222022


Origin: Module 6 – Speech Synthesis – waveform generation and connected speech
Translate + Edit: YangSier (Homepage)

:four_leaf_clover:碎碎念:four_leaf_clover:
Hello米娜桑,这里是英国留学中的杨丝儿。我的博客的关键词集中在编程、算法、机器人、人工智能、数学等等,点个关注吧,持续高质量输出中。
:cherry_blossom:唠嗑QQ群兔叽的魔术工房 (942848525)
:star:B站账号白拾Official(活跃于知识区和动画区)