Turning an unknown audio data stream into wav or similar format

I am trying to get the commentary (casters voice) from a dota2 game file. I’ve managed to parse the game file and select what I believe is the voice data. This is in a weird format (CSVCMsg_VoiceData) which has the following struc:

type CSVCMsg_VoiceData struct {
Client                   *int32            `protobuf:"varint,1,opt,name=client" json:"client,omitempty"`
Proximity                *bool             `protobuf:"varint,2,opt,name=proximity" json:"proximity,omitempty"`
Xuid                     *uint64           `protobuf:"fixed64,3,opt,name=xuid" json:"xuid,omitempty"`
AudibleMask              *int32            `protobuf:"varint,4,opt,name=audible_mask" json:"audible_mask,omitempty"`
VoiceData                []byte            `protobuf:"bytes,5,opt,name=voice_data" json:"voice_data,omitempty"`
Caster                   *bool             `protobuf:"varint,6,opt,name=caster" json:"caster,omitempty"`
Format                   *VoiceDataFormatT `protobuf:"varint,7,opt,name=format,enum=VoiceDataFormatT,def=1" json:"format,omitempty"`
SequenceBytes            *int32            `protobuf:"varint,8,opt,name=sequence_bytes" json:"sequence_bytes,omitempty"`
SectionNumber            *uint32           `protobuf:"varint,9,opt,name=section_number" json:"section_number,omitempty"`
UncompressedSampleOffset *uint32           `protobuf:"varint,10,opt,name=uncompressed_sample_offset" json:"uncompressed_sample_offset,omitempty"`
XXX_unrecognized         []byte            `json:"-"`

}

This seems to work when reading the data. Logically I’m probably looking for the VoiceData part of the struct when given this:

"format":0,"voice_data":"uz+ACgEAEAELgD4EQgEWAKV4mxnepfmhxKCQxAnKVNaHhKRXPIsmAH5RjXmJV0u+WTmrvgyCKxcraehjo/ZeKcFjksXQZEeOju4hLNv/MAB9KA7ww14Vc0ndYPB7dDXoXTexuxcW0Jg/diMgdH5ijWhe02Ch48KX86qJZYFyZV81AH76qCgh9AXliMdyWEgWTMbRD6xMX37WJALrXlSnxymIloSq2KGwXCcMXzQiSQIrcLVNfqdNJACCluFOIRKPmugUvsLZmnD04X0xhpAuNkwJECK4t51MBOWNWJlCAIDyZlJwWI45EPTjBB6yKyGOclu96qBV2MhFAh1d2J7WDZwe6YxOVu/BGkGcur9qTP85ZRfjANoiQxQrWvpoHFBFBy0AfX6k8XvbSwrk2nUAEP3P6kcmXORKUNKeu8HDnOUflQqtA5AkkTiun77fZrqnimIfWg==","sequence_bytes":23598094,"section_number":1,"sample_rate":16000

I’m able to pull the voice data out like so:
uz+ACgEAEAELgD4EQgEWAKV4mxnepfmhxKCQxAnKVNaHhKRXPIsmAH5RjXmJV0u+WTmrvgyCKxcraehjo/ZeKcFjksXQZEeOju4hLNv/MAB9KA7ww14Vc0ndYPB7dDXoXTexuxcW0Jg/diMgdH5ijWhe02Ch48KX86qJZYFyZV81AH76qCgh9AXliMdyWEgWTMbRD6xMX37WJALrXlSnxymIloSq2KGwXCcMXzQiSQIrcLVNfqdNJACCluFOIRKPmugUvsLZmnD04X0xhpAuNkwJECK4t51MBOWNWJlCAIDyZlJwWI45EPTjBB6yKyGOclu96qBV2MhFAh1d2J7WDZwe6YxOVu/BGkGcur9qTP85ZRfjANoiQxQrWvpoHFBFBy0AfX6k8XvbSwrk2nUAEP3P6kcmXORKUNKeu8HDnOUflQqtA5AkkTiun77fZrqnimIfWg==

However this is where I’m hitting a bit of a wall. This data is in an unknown format. I’ve tried to do some research on what the format might be and I’ve found that steam started using SILK codec for voice data in 2011 – however when trying to write this data to file and open it with opus (which I believe supports SILK) the opus decoder tells me it can’t open the file – so I’m not 100% convinced it is silk codec. Recognising audio data isn’t something I have a great deal of experience with – so any advice would be great.

I have noticed there’s a VoiceDataFormatT part of the struct but the only definition I can find for it is this:

type VoiceDataFormatT int32

Which doesn’t seem too helpful! :/

EDIT 1:
As per advice from user Ian Cook I’ve decoded the data from base64 into the following (as hex dump):

BB 3F 80 0A 01 00 10 01 0B 80 3E 04 42 01 16 00 A5 78 9B 19 DE A5 F9 A1 C4 A0 90 C4 09 CA 54 D6 87 84 A4 57 3C 8B 26 00 7E 51 8D 79 89 57 4B BE 59 39 AB BE 0C 82 2B 17 2B 69 E8 63 A3 F6 5E 29 C1 63 92 C5 D0 64 47 8E 8E EE 21 2C DB FF 30 00 7D 28 0E F0 C3 5E 15 73 49 DD 60 F0 7B 74 35 E8 5D 37 B1 BB 17 16 D0 98 3F 76 23 20 74 7E 62 8D 68 5E D3 60 A1 E3 C2 97 F3 AA 89 65 81 72 65 5F 35 00 7E FA A8 28 21 F4 05 E5 88 C7 72 58 48 16 4C C6 D1 0F AC 4C 5F 7E D6 24 02 EB 5E 54 A7 C7 29 88 96 84 AA D8 A1 B0 5C 27 0C 5F 34 22 49 02 2B 70 B5 4D 7E A7 4D 24 00 82 96 E1 4E 21 12 8F 9A E8 14 BE C2 D9 9A 70 F4 E1 7D 31 86 90 2E 36 4C 09 10 22 B8 B7 9D 4C 04 E5 8D 58 99 42 00 80 F2 66 52 70 58 8E 39 10 F4 E3 04 1E B2 2B 21 8E 72 5B BD EA A0 55 D8 C8 45 02 1D 5D D8 9E D6 0D 9C 1E E9 8C 4E 56 EF C1 1A 41 9C BA BF 6A 4C FF 39 65 17 E3 00 DA 22 43 14 2B 5A FA 68 1C 50 45 07 2D 00 7D 7E A4 F1 7B DB 4B 0A E4 DA 75 00 10 FD CF EA 47 26 5C E4 4A 50 D2 9E BB C1 C3 9C E5 1F 95 0A AD 03 90 24 91 38 AE 9F BE DF 66 BA A7 8A 62 1F 5A

I’m still at a loss as to what this information is – I’ve tried converting it to a wav file using ffmpeg (assuming is pcm) but it still comes out as white noise.

EDIT 2:
So it’s occurred to me that it might help if I include more samples of the data – the decoded hex of the data can be found here (each sample separated by a new line character):

pastebin

I’ve noticed that each one seems to start with the following hex:

BB 3F 80 0A 01 00 10 01 0B 80 3E 04

Which translates to:

»?€
�€>

I’m still at a loss as to how to convert this to audio data.

EDIT 3:
I’ve uploaded some more datadumps to the following pastebin (More data), it’s not a full dump as it’s roughly 15mb and pastebin crashed when I was trying to paste!

The data file is a dota2 demo file (extension .dem) which is a collection of protobuf messages that I parse using GoLang and the Manta replay parse (found here). This allows me to pull out any type of message, and I select OnCSVCMsg_VoiceData, which returns m.Audio.VoiceData of the form: CSVCMsg_VoiceData (the struct I display above).

EDIT 4

Here’s (finally) the link to the file with the concatenated voiceData messages.

And here’s the link to the original file of protobuff messages

import json
import base64
import sys

def main():
    if len(sys.argv) < 2:
        print("Usage: python3", sys.argv[0], "<CSVCMsg_VoiceData json file>")
        exit(1)

    with open(sys.argv[1], 'r') as infile:
        json_data = json.load(infile)

    # Create dictionary with section number as the key and list of
    # that section's structs as the value
    section_dict = {}
    for obj in json_data:
        sec_num = obj['section_number']
        if sec_num not in section_dict:
            section_dict[sec_num] = []
        section_dict[sec_num].append(obj)

    # Create SILK file for each section number stream
    for section in section_dict.keys():
        filename=f"section_{section}.slk"
        print(f"Generating SILK file {filename} for section {section}...")
        with open(filename, 'wb') as outfile:
            # SILK header
            outfile.write(b"#!SILK_V3")
            # Sort frames in ascending order based on sequence_bytes value
            for frame in sorted(section_dict[section], key=lambda x : x['sequence_bytes']):
                decoded = base64.b64decode(frame['voice_data'])
                # strip first 14 bytes and last 4 bytes before writing
                outfile.write(decoded[14:-4])

if __name__ == '__main__':
    main()

#!/bin/bash

# Write each decoded VoiceData to a file with the naming convention
# <sequence_bytes>_<section_number>
write_data ()
{
    filename=`echo $1 | cut -d_ -f1,2`
    data=`echo $1 | cut -d_ -f3`
    echo -n "$data" | base64 -d > $filename
}
export -f write_data
jq -r '.[] | "(.sequence_bytes)_(.section_number)_(.voice_data)"' dota2CasterParse.json | xargs -I '{}' bash -c "write_data '{}'"

#!/bin/bash

section_numbers=$(ls [0-9]*_[0-9]* | cut -d_ -f2 | sort -u)

for section in $section_numbers; do
    output="section_${section}_voiceData.slk"
    echo -n '#!SILK_V3' > $output
    for i in $(ls *_${section} | sort -n); do
        dd bs=1 skip=14 count=$(($(stat -c "%s" $i)-18)) if=$i of=$output conv=notrunc oflag=append
    done
done