Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom Voice Controller Example #67

Merged
merged 5 commits into from
Jul 29, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,16 @@ jobs:
xcode: << parameters.xcode >>
steps:
- checkout
- run:
name: Prepare .netrc file
command: |
echo "machine dl.bintray.com" > ~/.netrc
echo "login $BINTRAY_LOGIN" >> ~/.netrc
echo "password $BINTRAY_API_KEY" >> ~/.netrc
echo >> ~/.netrc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the end I think there is no need in entry for dl.bintray.com anymore because since nav. native version 14.1.5 and higher is distributed via SDK Registry.

echo "machine api.mapbox.com" >> ~/.netrc
echo "login mapbox" >> ~/.netrc
echo "password $SDK_REGISTRY_TOKEN" >> ~/.netrc
- run:
name: Install Dependencies
command: pod install
Expand Down
165 changes: 145 additions & 20 deletions Navigation-Examples/Examples/Custom-Voice-Controller.swift
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,14 @@ import UIKit
import MapboxCoreNavigation
import MapboxNavigation
import MapboxDirections
import MapboxSpeech
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to include this in order to be able to call SpeechSynthesizingDelegate methods (needs access to SpeechOptions constructor). This makes me wonders if this mechanism could be enhanced to avoid importing additional modules.

import AVFoundation

class CustomVoiceControllerUI: UIViewController {

var voiceController: CustomVoiceController?

override func viewDidLoad() {
super.viewDidLoad()


let origin = CLLocationCoordinate2DMake(37.77440680146262, -122.43539772352648)
let destination = CLLocationCoordinate2DMake(37.76556957793795, -122.42409811526268)
let routeOptions = NavigationRouteOptions(coordinates: [origin, destination])
Expand All @@ -28,47 +26,174 @@ class CustomVoiceControllerUI: UIViewController {

// For demonstration purposes, simulate locations if the Simulate Navigation option is on.
let navigationService = MapboxNavigationService(route: route, routeOptions: routeOptions, simulating: simulationIsEnabled ? .always : .onPoorGPS)
strongSelf.voiceController = CustomVoiceController(navigationService: navigationService)
let navigationOptions = NavigationOptions(navigationService: navigationService, voiceController: strongSelf.voiceController)

// `MultiplexedSpeechSynthesizer` will provide "a backup" functionality to cover cases, which
// our custom implementation cannot handle.
let speechSynthesizer = MultiplexedSpeechSynthesizer([CustomVoiceController(), SystemSpeechSynthesizer()] as? [SpeechSynthesizing])
let routeController = RouteVoiceController(navigationService: navigationService, speechSynthesizer: speechSynthesizer)
// Remember to pass our `Voice Controller` to `Navigation Options`!
let navigationOptions = NavigationOptions(navigationService: navigationService, voiceController: routeController)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe better to rename routeController to routeVoiceController?


let navigationViewController = NavigationViewController(for: route, routeOptions: routeOptions, navigationOptions: navigationOptions)
navigationViewController.modalPresentationStyle = .fullScreen

strongSelf.present(navigationViewController, animated: true, completion: nil)
}
}
}

override func viewWillDisappear(_ animated: Bool) {
super.viewWillDisappear(animated)
voiceController = nil
}
}

class CustomVoiceController: MapboxVoiceController {
class CustomVoiceController: NSObject, SpeechSynthesizing {

// MARK: - SpeechSynthesizing implementation

var delegate: SpeechSynthesizingDelegate?

public var muted: Bool = false {
didSet {
updatePlayerVolume(audioPlayer)
}
}
public var volume: Float = 1.0 {
didSet {
audioPlayer?.volume = volume
}
}
public var isSpeaking: Bool {
return audioPlayer?.isPlaying ?? false
}

var locale: Locale?

func prepareIncomingSpokenInstructions(_ instructions: [SpokenInstruction], locale: Locale?) {
// do nothing as we don't have to prepare anything
}

func stopSpeaking() {
audioPlayer?.stop()
}

func interruptSpeaking() {
audioPlayer?.stop()
}

// You will need audio files for as many or few cases as you'd like to handle
// This example just covers left, right and straight.
// This example just covers left and right. All other cases will fail the Custom Voice Controller and force a backup System Speech to kick in
let turnLeft = NSDataAsset(name: "turnleft")!.data
let turnRight = NSDataAsset(name: "turnright")!.data
let straight = NSDataAsset(name: "continuestraight")!.data

override func didPassSpokenInstructionPoint(notification: NSNotification) {
let routeProgress = notification.userInfo![RouteController.NotificationUserInfoKey.routeProgressKey] as! RouteProgress
let soundForInstruction = audio(for: routeProgress.currentLegProgress.currentStep)
let instruction = notification.userInfo![RouteController.NotificationUserInfoKey.spokenInstructionKey] as! SpokenInstruction
play(instruction: instruction, data: soundForInstruction)
public var audioPlayer: AVAudioPlayer?
private var previousInstruction: SpokenInstruction?

func speak(_ instruction: SpokenInstruction, during legProgress: RouteLegProgress, locale: Locale? = nil) {

guard let soundForInstruction = audio(for: legProgress.currentStep) else {
// When `MultiplexedSpeechSynthesizer` receives an error from one of it's Speech Synthesizers,
// it requests the next on the list
delegate?.speechSynthesizer(self,
didSpeak: instruction,
with: SpeechError.noData(instruction: instruction,
options: SpeechOptions(text: instruction.text)))
return
}
speak(instruction: instruction, instructionData: soundForInstruction)
}

func audio(for step: RouteStep) -> Data {
func audio(for step: RouteStep) -> Data? {
switch step.maneuverDirection {
case .left:
return turnLeft
case .right:
return turnRight
default:
return straight
return nil // this will force report that Custom View Controller is unable to handle this case
}
}

// Method to play provided audio data with some edge cases handling
func speak(instruction: SpokenInstruction, instructionData: Data) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same method in MapboxSpeechSynthesizer is iternal. Having it public would've make this example much shorter. Do you think it is safe to expose it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not have fair amount of experience of working with speech instructions, but seems that it should be fine to expose this method publicly. E.g. it should align well in case when user creates spokenInstruction manually and then wants to speak it whenever needed. On the other hand it's not obvious what is instructionData and how to create/use it correctly.


if let audioPlayer = audioPlayer {
if let previousInstruction = previousInstruction, audioPlayer.isPlaying {
delegate?.speechSynthesizer(self,
didInterrupt: previousInstruction,
with: instruction)
}

deinitAudioPlayer()
}

switch safeInitializeAudioPlayer(data: instructionData,
instruction: instruction) {
case .success(let player):
audioPlayer = player
previousInstruction = instruction
audioPlayer?.play()
case .failure(let error):
safeUnduckAudio(instruction: instruction)
delegate?.speechSynthesizer(self,
didSpeak: instruction,
with: error)
}
}

// MARK: - Audio control methods

func safeDuckAudio(instruction: SpokenInstruction?){
if let error = AVAudioSession.sharedInstance().tryDuckAudio() {
delegate?.speechSynthesizer(self,
encounteredError: SpeechError.unableToControlAudio(instruction: instruction,
action: .duck,
underlying: error))
}
}

func safeUnduckAudio(instruction: SpokenInstruction?) {
if let error = AVAudioSession.sharedInstance().tryUnduckAudio() {
delegate?.speechSynthesizer(self,
encounteredError: SpeechError.unableToControlAudio(instruction: instruction,
action: .unduck,
underlying: error))
}
}

func updatePlayerVolume(_ player: AVAudioPlayer?) {
player?.volume = muted ? 0.0 : volume
}

func safeInitializeAudioPlayer(data: Data, instruction: SpokenInstruction) -> Result<AVAudioPlayer, MapboxNavigation.SpeechError> {
do {
let player = try AVAudioPlayer(data: data)
player.delegate = self
updatePlayerVolume(player)

return .success(player)
} catch {
return .failure(SpeechError.unableToInitializePlayer(playerType: AVAudioPlayer.self,
instruction: instruction,
synthesizer: nil,
underlying: error))
}
}

func deinitAudioPlayer() {
audioPlayer?.stop()
audioPlayer?.delegate = nil
}
}

extension CustomVoiceController: AVAudioPlayerDelegate {
public func audioPlayerDidFinishPlaying(_ player: AVAudioPlayer, successfully flag: Bool) {
safeUnduckAudio(instruction: previousInstruction)

guard let instruction = previousInstruction else {
assert(false, "Speech Synthesizer finished speaking 'nil' instruction")
return
}

delegate?.speechSynthesizer(self,
didSpeak: instruction,
with: nil)
}
}

4 changes: 2 additions & 2 deletions Podfile
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
platform :ios, '10.0'
use_frameworks!

pod 'MapboxCoreNavigation', :git => 'https://github.com/mapbox/mapbox-navigation-ios.git', :tag => 'v1.0.0-beta.1'
pod 'MapboxNavigation', :git => 'https://github.com/mapbox/mapbox-navigation-ios.git', :tag => 'v1.0.0-beta.1'
pod 'MapboxCoreNavigation', :git => 'https://github.com/mapbox/mapbox-navigation-ios.git', :branch => 'release-v1.0-pre-registry'
pod 'MapboxNavigation', :git => 'https://github.com/mapbox/mapbox-navigation-ios.git', :branch => 'release-v1.0-pre-registry'

target 'Navigation-Examples' do
end
Expand Down
36 changes: 18 additions & 18 deletions Podfile.lock
Original file line number Diff line number Diff line change
@@ -1,32 +1,32 @@
PODS:
- Mapbox-iOS-SDK (5.9.0):
- Mapbox-iOS-SDK (6.0.0):
- MapboxMobileEvents (= 0.10.2)
- MapboxAccounts (2.3.0)
- MapboxCoreNavigation (1.0.0-beta.1):
- MapboxAccounts (~> 2.3.0)
- MapboxDirections (~> 0.32.0)
- MapboxDirections (~> 0.33.0)
- MapboxMobileEvents (~> 0.10.2)
- MapboxNavigationNative (~> 14.1.5)
- Turf (~> 0.5.0)
- MapboxDirections (0.32.0):
- MapboxDirections (0.33.0):
- Polyline (~> 4.2)
- Turf (~> 0.5)
- MapboxMobileEvents (0.10.2)
- MapboxNavigation (1.0.0-beta.1):
- Mapbox-iOS-SDK (~> 5.6)
- Mapbox-iOS-SDK (~> 6.0)
- MapboxCoreNavigation (= 1.0.0-beta.1)
- MapboxMobileEvents (~> 0.10.2)
- MapboxSpeech (~> 0.3.0)
- Solar (~> 2.1)
- MapboxNavigationNative (14.1.5)
- MapboxNavigationNative (14.1.6)
- MapboxSpeech (0.3.0)
- Polyline (4.2.1)
- Solar (2.1.0)
- Turf (0.5.0)

DEPENDENCIES:
- MapboxCoreNavigation (from `https://github.com/mapbox/mapbox-navigation-ios.git`, tag `v1.0.0-beta.1`)
- MapboxNavigation (from `https://github.com/mapbox/mapbox-navigation-ios.git`, tag `v1.0.0-beta.1`)
- MapboxCoreNavigation (from `https://github.com/mapbox/mapbox-navigation-ios.git`, branch `release-v1.0-pre-registry`)
- MapboxNavigation (from `https://github.com/mapbox/mapbox-navigation-ios.git`, branch `release-v1.0-pre-registry`)

SPEC REPOS:
trunk:
Expand All @@ -42,33 +42,33 @@ SPEC REPOS:

EXTERNAL SOURCES:
MapboxCoreNavigation:
:branch: release-v1.0-pre-registry
:git: https://github.com/mapbox/mapbox-navigation-ios.git
:tag: v1.0.0-beta.1
MapboxNavigation:
:branch: release-v1.0-pre-registry
:git: https://github.com/mapbox/mapbox-navigation-ios.git
:tag: v1.0.0-beta.1

CHECKOUT OPTIONS:
MapboxCoreNavigation:
:commit: bb3b5bcee12edeb5c6422121186a7a1010e58d2a
:git: https://github.com/mapbox/mapbox-navigation-ios.git
:tag: v1.0.0-beta.1
MapboxNavigation:
:commit: bb3b5bcee12edeb5c6422121186a7a1010e58d2a
:git: https://github.com/mapbox/mapbox-navigation-ios.git
:tag: v1.0.0-beta.1

SPEC CHECKSUMS:
Mapbox-iOS-SDK: a5915700ec84bc1a7f8b3e746d474789e35b7956
Mapbox-iOS-SDK: 48ff7550bd459b8494da8ef4d660a668e9187ef0
MapboxAccounts: 84abfdde95d9dc483f604c1b0fe1861edf691ce7
MapboxCoreNavigation: f6ee6f4093a5fd02619bd9d3fc35b1e8cefe61b2
MapboxDirections: 7f36b3e9ef6a53fc997c114a341ab4da721756bd
MapboxCoreNavigation: 3dc1accaca3a876238f89af294460340ae34fae9
MapboxDirections: dd01b8f9053719e02d87538b68355d3206321c81
MapboxMobileEvents: 2bc0ca2eedb627b73cf403258dce2b2fa98074a6
MapboxNavigation: 041ea970d7355db6901eec275aa6430760760d4e
MapboxNavigationNative: a4bb15f37b174b3bd998bb06e745bad64d4bcefa
MapboxNavigation: e031095fa956e6a4af90e57577392709e3826da0
MapboxNavigationNative: 05d47617f3d18c5e45d9b549cb82dc39cdb67693
MapboxSpeech: 403415e932e084cf290b9d55c49ab7ea210b9595
Polyline: 0e9890790292741c8186201a536b6bb6a78d02dd
Solar: 2dc6e7cc39186cb0c8228fa08df76fb50c7d8f24
Turf: bdcfefe60df3ca8dc5f742681c178e47f8c5a567

PODFILE CHECKSUM: 2c490b8d8a523b942db04bffc53c02571567ca9c
PODFILE CHECKSUM: cec58a1a1d1528bd6c0191996fc41052fadf3ed7

COCOAPODS: 1.9.3
COCOAPODS: 1.8.4