Typolino – Adding Speech

If we want to add sound to Typolino we need to consider the proper format. I’m not an expert but https://caniuse.com is usually very good at answering these kind of questions and also Wikipedia provides a good overview. I don’t care too much about the lossyness of the format as my audio capture device anyhow is pretty cheap. So let’s try with mp3.

To record the audio I usually use Audacity. The tool is great and full with features. To make it a bit more complicated to code let’s try to keep one audio file and just jump to the correct location for every character. The map needs to be confiugred and only one audio resource needs to be managed. This would be a similar approach to what we use to do with CSS sprites. Having zero to no experience I would say: just try it. 🙂

Recording

Recording was quite straight forward. Just had to find a moment of (almost) silence. Audacity is really cool – they have so many built in effects. I was only interested in the one to remove silence and the noise suppression. I think especially noise suppression cleans the curves quite a bit and helps in keeping the file size smaller. But really – not an expert. 🙂

The exported mp3 file is roughly 200k. Not sure what else we can do to improve – but for now I’m quite happy. Without noise suppression it was around 260k.

Storing

I keep the file as an asset. For this I just put it into the assets folder.

Seek and Play

For our audio sprite we need a service. It seems to be a bit cumbersome to use the native API so I found this Angular *wrapper* which looks quite promising:
https://github.com/chrisguttandin/angular-audio-context

Installation is simple:

 npm install --save angular-audio-context 

…and it looks way too complex. I don’t want to deal with all the details of the web audio API. But as I’m writing as I’m coding sometimes you change your mind. Not sure how I found this one (but it was not obvious to me at least), but it looks so much easier and seems to offer exactly what I need:
https://howlerjs.com/

It even supports Audio sprites, which.. kind of answers my initial question. 🙂 So let’s try this one.

 npm install --save howler 

import { Howl, Howler } from 'howler';
import { Injectable } from '@angular/core';

@Injectable({
  providedIn: 'root',
})
export class AudioService {
  private sound: Howl;

  constructor() {
    this.sound = new Howl({
      src: ['assets/alphabet.mp3'],
      sprite: {
        A: [0, 380],
        B: [389, 450],
        C: [830, 530],
        D: [1360, 420],
        E: [1805, 460],
        F: [2280, 360],
        G: [2650, 500],
        H: [3115, 490],
        I: [3615, 460],
        J: [4065, 360],
        K: [4440, 470],
        L: [4920, 510],
        M: [5430, 480],
        N: [5920, 530],
        O: [6460, 490],
        P: [6960, 460],
        Q: [7427, 510],
        R: [7937, 465],
        S: [8413, 557],
        T: [8982, 495],
        U: [9467, 470],
        V: [9950, 470],
        W: [10439, 528],
        X: [10972, 560],
        Y: [11548, 770],
        Z: [12380, 365],
      },
    });
  }

  play(letter: string) {
    this.sound.play(letter.toUpperCase());
  }
}

Conclusion

It works, not sure how well it will work in the wild. But it was interesting to implement this functionality. It took a bit of time in setting up the sprite but howlerjs just worked out of the box. Just for the sake of completeness: very first I was thinking of using speech synthesis API but I wasn’t so happy with the results.