Preface
I’ve been studying the
AV Foundationframework recently and found a book called AV Foundation开发秘籍：实践掌握iOS & OS X 应用的视听处理技术 Then I Googled and found the English version is called Learning AV Foundation: A Hands-on Guide to Mastering the AV Foundation Framework Reading the Chinese translation, I couldn’t help but sigh: why not just write a book yourself instead of going through all the trouble of translating it, turning something originally full of technical substance into something that, through literal translation, loses all its technical value. Seeing the title “开发秘籍” (Development Secrets), I can’t help but think of those books from my college days — from “Development for Beginners” to “Learn xxx in 21 Days” to “Development Guide” to “Development Secrets”… I readfake booksin college.
Today I’m sharing how to convert Chinese characters into speech on iOS. There’s not much technical content to this (experts can skip it).
AVFoundation Overall Architecture
Before diving into this feature, let me introduce the overall architecture of AV Foundation.
This is the architecture design on iOS (above).
This is the architecture design on macOS (above).
Now let’s implement this demo with code. First, import <AVFoundation/AVFoundation.h>
Here I need to use AVSpeechSynthesizer on iOS, which is called NSSpeechSynthesizer on macOS.
1
@property (strong, nonatomic) AVSpeechSynthesizer *synthesizer;
AVSpeechSynthesizer — its functionality:
- Add text to speech, i.e., play a piece of text as speech
Initialization
1
2
3
4
5
6
7
8
9
10
11
- (void)awakeFromNib {
[super awakeFromNib];
// Create speech synthesizer
self.synthesizer = [[AVSpeechSynthesizer alloc] init];
self.synthesizer.delegate = self;
// Languages for playback
self.voices = @[[AVSpeechSynthesisVoice voiceWithLanguage:@"zh-CN"],[AVSpeechSynthesisVoice voiceWithLanguage:@"en-US"]
];
self.speechStrings = [[NSMutableArray alloc] init];
}
Here, [AVSpeechSynthesisVoice voiceWithLanguage:@"zh-CN"] sets up Simplified Chinese speech. A full list of supported speech voices will be provided at the end of the article, so don’t worry about getting it wrong.
The delegate methods of AVSpeechSynthesizer are as follows — mainly for monitoring speech playback status:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
@protocol AVSpeechSynthesizerDelegate <NSObject>
// Delegate methods
@optional
// Did start speech utterance
- (void)speechSynthesizer:(AVSpeechSynthesizer *)synthesizer didStartSpeechUtterance:(AVSpeechUtterance *)utterance;
// Did finish speech utterance
- (void)speechSynthesizer:(AVSpeechSynthesizer *)synthesizer didFinishSpeechUtterance:(AVSpeechUtterance *)utterance;
// Did pause speech utterance
- (void)speechSynthesizer:(AVSpeechSynthesizer *)synthesizer didPauseSpeechUtterance:(AVSpeechUtterance *)utterance;
// Did continue speech utterance
- (void)speechSynthesizer:(AVSpeechSynthesizer *)synthesizer didContinueSpeechUtterance:(AVSpeechUtterance *)utterance;
// Did cancel speech utterance
- (void)speechSynthesizer:(AVSpeechSynthesizer *)synthesizer didCancelSpeechUtterance:(AVSpeechUtterance *)utterance;
// Used to monitor the character range being spoken
- (void)speechSynthesizer:(AVSpeechSynthesizer *)synthesizer willSpeakRangeOfSpeechString:(NSRange)characterRange utterance:(AVSpeechUtterance *)utterance;
@end
The main methods of AVSpeechSynthesizer are:
1
2
3
4
5
6
7
8
9
10
/* Add a speech utterance to the speech queue. You can control playback by setting the utterance's properties. */
- (void)speakUtterance:(AVSpeechUtterance *)utterance;
// For stopSpeakingAtBoundary: operations on speech utterances, if interrupted, the queue will be cleared
// Interrupt
- (BOOL)stopSpeakingAtBoundary:(AVSpeechBoundary)boundary;
// Pause
- (BOOL)pauseSpeakingAtBoundary:(AVSpeechBoundary)boundary;
// Resume
- (BOOL)continueSpeaking;
Here we use the
speakUtterancemethod to play text. speakUtterance:(AVSpeechUtterance *)utterance
AVSpeechUtteranceis an encapsulation of text for speech playback- The speech text to be played, which can be understood as a piece of text that needs to be played Here we set the
AVSpeechUtteranceplayback information:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
// Play speech
NSArray *speechStringsArray = [self buildSpeechStrings]; // buildSpeechStrings returns an array of speech strings
for (NSUInteger i = 0; i < speechStringsArray.count; i++) {
// Create AVSpeechUtterance object for the speech text to be played
AVSpeechUtterance *utterance = [[AVSpeechUtterance alloc] initWithString:speechStringsArray[i]];
// Set which language to use for playback
utterance.voice = self.voices[0];
// The speech rate for this text, should be between AVSpeechUtteranceMinimumSpeechRate and AVSpeechUtteranceMaximumSpeechRate
utterance.rate = 0.5;
// Change the pitch when playing specific sentences, typically between 0.5 (low pitch) ~ 2.0 (high pitch)
utterance.pitchMultiplier = 0.8f;
// Volume, between 0.0 ~ 1.0
utterance.volume = 1.0f;
// Delay after playback, i.e., the pause time after this text finishes playing, default is 0
utterance.preUtteranceDelay = 0;
// Delay before playback, i.e., the pause time before this text starts playing, default is 0
utterance.postUtteranceDelay = 0.1f;
[self.synthesizer speakUtterance:utterance];
}
Properties of AVSpeechUtterance:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// Set which language to use for playback
@property(nonatomic, retain, nullable) AVSpeechSynthesisVoice *voice;
// Get the text to be played, read-only property
@property(nonatomic, readonly) NSString *speechString;
// Get the text to be played as attributed string, read-only, available from iOS 10
@property(nonatomic, readonly) NSAttributedString *attributedSpeechString;
// The speech rate for this text, should be between AVSpeechUtteranceMinimumSpeechRate and AVSpeechUtteranceMaximumSpeechRate
@property(nonatomic) float rate;
// Change the pitch when playing specific sentences, typically between 0.5 (low pitch) ~ 2.0 (high pitch)
@property(nonatomic) float pitchMultiplier;
// Volume, between 0.0 ~ 1.0
@property(nonatomic) float volume;
// Delay after playback, i.e., the pause time after this text finishes playing, default is 0
@property(nonatomic) NSTimeInterval preUtteranceDelay;
// Delay before playback, i.e., the pause time before this text starts playing, default is 0
@property(nonatomic) NSTimeInterval postUtteranceDelay;
Methods of AVSpeechUtterance:
The following are all initialization methods, divided into class methods and instance methods. The attributed string initialization method is only available from iOS 10:
1
2
3
4
5
+ (instancetype)speechUtteranceWithString:(NSString *)string;
+ (instancetype)speechUtteranceWithAttributedString:(NSAttributedString *)string NS_AVAILABLE_IOS(10_0);
- (instancetype)initWithString:(NSString *)string;
- (instancetype)initWithAttributedString:(NSAttributedString *)string
You can use [AVSpeechSynthesisVoice speechVoices] to print out all supported speech languages:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
ar-SA 沙特阿拉伯（阿拉伯文）
en-ZA, 南非（英文）
nl-BE, 比利时（荷兰文）
en-AU, 澳大利亚（英文）
th-TH, 泰国（泰文）
de-DE, 德国（德文）
en-US, 美国（英文）
pt-BR, 巴西（葡萄牙文）
pl-PL, 波兰（波兰文）
en-IE, 爱尔兰（英文）
el-GR, 希腊（希腊文）
id-ID, 印度尼西亚（印度尼西亚文）
sv-SE, 瑞典（瑞典文）
tr-TR, 土耳其（土耳其文）
pt-PT, 葡萄牙（葡萄牙文）
ja-JP, 日本（日文）
ko-KR, 南朝鲜（朝鲜文）
hu-HU, 匈牙利（匈牙利文）
cs-CZ, 捷克共和国（捷克文）
da-DK, 丹麦（丹麦文）
es-MX, 墨西哥（西班牙文）
fr-CA, 加拿大（法文）
nl-NL, 荷兰（荷兰文）
fi-FI, 芬兰（芬兰文）
es-ES, 西班牙（西班牙文）
it-IT, 意大利（意大利文）
he-IL, 以色列（希伯莱文，阿拉伯文）
no-NO, 挪威（挪威文）
ro-RO, 罗马尼亚（罗马尼亚文）
zh-HK, 香港（中文）
zh-TW, 台湾（中文）
sk-SK, 斯洛伐克（斯洛伐克文）
zh-CN, 中国（中文）
ru-RU, 俄罗斯（俄文）
en-GB, 英国（英文）
fr-FR, 法国（法文）
hi-IN 印度（印度文）
Summary To learn
AVFoundation, I started with a simple concept. The only regret is that I’m not sure whether this synthesizer supports custom speech playback. I’ll research this further and fill in the related learning content.
Final demo supporting both iOS and macOS: Learning-AV-Foundation (Part 1) Chinese Text-to-Speech
References: AV Foundation Apple Official Documentation AVSpeechSynthesizer and AVSpeechUtterance AVSpeechSynthesizer详解 AVFoundation