How to read list of UnsignedInteger16 from base64 string in mathematica

Question

I am struggling to convert a base64 string to a list of UnsignedInteger16 values.
I have limited experience with mathematica, so please excuse me if this should be obvious.

I read the base64 string from an XML file and ultimately into a variable base64String.
This is a long string with 7057 UnsignedInteger16 values, but it starts off as: "8ynnKdspzinCKbcprSmhKZUpiCl8KW8pYylXKUkpQCk0KSop…"

Can convert to ByteArry with n = BaseDecode[base64String] // Normal;

Then n has the expected values {243, 41, 231, 41, 219, 41, 206, 41, 194, 41, 183, 41, 173, 41, 161, …

However, what I am looking for is a list with the values {243 + 256 * 41, 231 + 256 * 41, 219 + 256 * 41, …

i.e.: {10739, 10727, 10727, …

Thank you in advance for you suggestions.

Edit: Thank you for the suggestion to use Partition. Works well. For the interests in improving my understanding, can any one suggest why I cannot get ImportString[…] to work.

My source data is XML file with data along the lines of the following where I truncated the Base64 string for clarity.:

<?xml version="1.0" encoding="utf-8" ?>
<BPplus version="5.0" filename="BPplus_00100.xml">
    <MeasDataLogger guid="ee7bee36-ffd6-30ae-53f1-257f0eab7ddd">
        <Sys>130</Sys>
        <Dia>77</Dia>
        <Map>101</Map>
        <PressureWaves>
            <RawPressureWave>
                <RawPressure>NIBP</RawPressure>
                <RawSampleCount>7057</RawSampleCount>
                <RawCuffPressureWave>8ynnKdspzinCKbcprSmhKZUpiCl8KW8pYylXKUk... bU1mjV+NWM1RjU=</RawCuffPressureWave>
            </RawPressureWave>
        </PressureWaves>
    </MeasDataLogger>
</BPplus>

The script I then run with the C1, C2 & C3 suggestions.

sampleFilenameString = "C:\BPPdata\BPplus_00112.xml";
xmldata = Import[sampleFilenameString, "XML"];
RawCuffPressureWave = 
  Cases[xmldata, XMLElement["RawCuffPressureWave", _, _], Infinity];
nibpxml = RawCuffPressureWave[[2]];
base64String = ToString[nibpxml[[3]]];
C1 = Partition[Normal@BaseDecode[base64String], 2].{1, 256};
C2 = ImportByteArray[BaseDecode[base64String], "UnsignedInteger16"];
C3 = ImportString[base64String, {"Base64", "UnsignedInteger16"}, 
  ByteOrdering -> -1]
C1 == C2
C1 == C3
C1

produces the following output

{17956, 26977, 25964}
True
False
{10739, 10727, 10715, 10702, 10690, 10679, 10669, 10657, ... }

I have tried to make the first parameter ToString[base64String], but that did not change the outcome.

If I manually define the string $base64 = "8ynnKdspzinCKbcprSmhKZUpiCl8KW8pYylXKUkpQCk0KSo....

It works as expected and C1 = C4

C4 = ImportString[$base64, {"Base64", "UnsignedInteger16"}, 
   ByteOrdering -> -1];
C1 == C4

Can anyone explain the difference why $base64 variable works but base64String does not?

Edit: base64String is not a string! Not sure if this is the best way to convert, but it works in Mathematica 11.x and higher.

sampleFilenameString = "C:\BPPdata\BPplus_00112.xml";
xmldata = Import[sampleFilenameString, "XML"];
RawCuffPressureWave = Cases[xmldata, XMLElement["RawCuffPressureWave", _, _], Infinity];
nibpxml = RawCuffPressureWave[[2]];
(* convert to string then base64 decode *)
base64Data = 
  Developer`DecodeBase64ToByteArray[
     nibpxml //. XMLElement[_, _, t_] :> t // Flatten // StringJoin]  // Normal;
C1 = Partition[base64Data, 2].{1, 256};

import xml

Roman · Answer

There are many ways of doing this. The fastest I can think of is
Partition[Normal@BaseDecode[str], 2].{1, 256}

which takes less than half of the time that ImportString requires.
Timing comparisons:
A = RandomInteger[255, 10^8] // ByteArray;
S = BaseEncode[A];

C1 = Partition[Normal@BaseDecode[S], 2].{1, 256}; // AbsoluteTiming // First
(*    4.11495    *)

C2 = ImportByteArray[BaseDecode[S], "UnsignedInteger16"]; // AbsoluteTiming // First
(*    5.98029    *)    (*    @ilian    *)

C3 = ImportString[S, {"Base64", "UnsignedInteger16"}, ByteOrdering -> -1]; // AbsoluteTiming // First
(*    9.64049    *)    (*    @WReach    *)

C1 == C2 == C3
(*    True    *)

WReach · Answer

We can use ImportString to decode the Base64 string and interpret the resulting bytes in a single step:
ImportString["8ynnKdsp", {"Base64", "UnsignedInteger16"}, ByteOrdering -> -1]
(* {10739, 10727, 10715} *)

Explanation
The documentation for the Base64 import format tells us:

The import format and file encoding can be explicitly specified with Import["file", {"Base64", "format", elems1, elems2, ...].

In this case, format can be "Binary" which is documented to support many data representation elements including "UnsignedInteger16".
Putting all this together with ImportString, we can perform the required conversion:
$base64 = "8ynnKdspzinCKbcprSmhKZUpiCl8KW8pYylXKUkpQCk0KSop";

ImportString[$base64, {"Base64", "Binary", "UnsignedInteger16"}, ByteOrdering -> -1]

(* {10739, 10727, 10715, 10702, 10690, 10679, 10669, 10657, 10645,
    10632, 10620, 10607, 10595, 10583, 10569, 10560, 10548, 10538} *)

The documentation for Binary also tells us:

For any data representation element type, Import["file", type] can be used as a shorthand for Import["file", {"Binary", type}].

So we can use this shorthand form to get the same result:
ImportString[$base64, {"Base64", "UnsignedInteger16"}, ByteOrdering -> -1]

The ByteOrdering option ensures that the unsigned integers are interpreted in "little-endian" format (low-byte first).  This option can be omitted on a little-endian machine since it will be the default $ByteOrdering.  But even so it does not hurt to include it defensively if there is any possibility that our code might some day find its way onto a big-endian machine.

How to read list of UnsignedInteger16 from base64 string in mathematica

2 Answers

Timing comparisons:

Add your own answers!

Ask a Question