Tech and Media Labs
This site uses cookies to improve the user experience.




IonWriter

Jakob Jenkov
Last update: 2015-12-18

The IonWriter class (com.jenkov.iap.ion.write.IonWriter) in IAP Tools can write ION fields to a byte array. With the IonWriter you can write ION fields freely, so it is up to you to make sure you follow the rules for the ION fields you write. This also means that you can break the rules in case you need that in a closed environment (e.g. for a specific data file, or for messages exchanged between two services which are not open to the public).

The IonWriter contains two sets of methods. The first set is its instance methods. To use these methods you must create an instance of IonWriter. The second set is the IonWriter's static methods. These can be called directly without an IonWriter instance.

The IonWriter instance methods are easier to use than the static methods. When you create an instance of IonWriter you set a byte array as destination for all ION fields written. The IonWriter keeps track of the index into the destination byte array for you. The static methods need all relevant information passed as parameters, like the destination byte array, the offset into this array to write the ION field to, and the value to write.

The following sections will explain both the instance methods and static methods of the IonWriter.

IonWriter Instance Methods

First we will take a look at the IonWriter's instance methods, since they are the easiest to use.

Creating an IonWriter

To use the IonWriter instance methods you must first create an instance of IonWriter and attach it to a destination byte array (into which the ION fields are written). Here is how you create an instance of IonWriter:

byte[]    dest   = new byte[10 * 1024];

IonWriter writer = new IonWriter(dest, 0);

This example creates an IonWriter instance, passing the destination byte array and start offset as parameters to the constructor. All ION fields will be written into the given byte array from the given offset and onwards.

You can also set the destination byte array on an IonWriter using the setDestination() method, like this:

byte[]    dest   = new byte[10 * 1024];

IonWriter writer = new IonWriter();

writer.setDestination(dest, 0);

Once a destination has been set on the IonWriter you can start writing ION fields to the destination byte array.

writeBytes()

The writeBytes() method writes a sequence of raw bytes as an ION Bytes field to the destination byte array. Here is an IonWriter writeBytes() example:

byte[] value = new byte[]{ 1, 2, 3 };
    
writer.writeBytes(value);

The writeBytes() method also supports writing null arrays out, in which case an ION Bytes field with the length-length 0 will be written (which represents a null Bytes field).

writeBoolean()

The writeBoolean() method writes a Java boolean value as an ION Tiny field to the destination byte array. Here is an IonWriter writeBoolean() example:

boolean value = true;

writer.writeBoolean(value);

writeBooleanObj()

The writeBooleanObj() method writes a Java Boolean object as an ION Tiny field to the destination byte array. Here is an IonWriter writeBooleanObj() example:

Boolean value = new Boolean(true);

writer.writeBooleanObj(value);

The writeBooleanObj() method also supports writing null out, in which case an ION Tiny field with the length-length 0 will be written (which represents a null Tiny field).

writeInt64()

The writeInt() method writes a Java long (or other integer) as an Int64-Positive or Int64-Negative field to the destination byte array. Here is an IonWriter writeInt64() example:

long value = 123456;

writer.writeInt64(value);

writeInt64Obj()

The writeInt64Obj() method writes a Java Long object as an Int64-Positive or Int64-Negative field to the destination byte array. Here is an IonWriter writeInt64() example:

Long value = new Long(123456);

writer.writeInt64Obj(value);

The writeInt64Obj() method also supports writing null out, in which case an ION Int64-Positive field with the length-length 0 will be written (which represents a null Int64-Positive field).

writeFloat32()

The writeFloat32() method writes a Java float as an ION Float field to the destination byte array. Here is an IonWriter writeFloat32() example:

float value = 123.123f;

writer.writeFloat32(value);

writeFloat32Obj()

The writeFloat32Obj() method writes a Java Float object as an ION Float field to the destination byte array. Here is an IonWriter writeFloat32Obj() example:

Float value = new Float(123.123f);

writer.writeFloat32Obj(value);

The writeFloat32Obj() method also supports writing null out, in which case an ION Float field with the length-length 0 will be written (which represents a null Float field).

writeFloat64()

The writeFloat64() method writes a Java double as an ION Float field to the destination byte array. Here is an IonWriter writeFloat64() example:

double value = 123.123d;

writer.writeFloat64(value);

writeFloat64Obj()

The writeFloat64Obj() method writes a Java Double object as an ION Float field to the destination byte array. Here is an IonWriter writeFloat64Obj() example:

Double value = new Double(123.123d);

writer.writeFloat64Obj(value);

The writeFloat64Obj() method also supports writing null out, in which case an ION Float field with the length-length 0 will be written (which represents a null Float field).

writeUtf8()

The writeUtf8() methods write a Java String or a byte array of UTF-8 encoded bytes as an ION UTF-8 or UTF-8-Short Field to the destination byte array. If there are 15 or less bytes in the UTF-8 bytes to write, an UTF-8-Short field will be written. Otherwise a UTF-8 field will be written.

Here are two IonWriter writeUtf8() examples:

String value = "Hello";

writer.writeUtf8(value);

byte[] valueBytes = "Hello".getBytes("UTF-8") ;

writer.writeUtf8(valueBytes);

Both of the writeUtf8() methods also supports writing null out, in which case an ION UTF-8 field with the length-length 0 will be written (which represents a null UTF-8 field).

If the String or byte array passed as parameter has 0 characters / elements, an UTF-8 field will be written with a single length byte with the value 0 (two bytes in total - 1 lead byte + 1 length byte). That is how an empty string is distinguished from a null value.

writeUtc()

The writeUtc() method writes a Java Calendar as an ION UTC field to the given byte array. Here is a IonWriter writeUtc() example:

byte[] dest = new byte[10 * 1024];
int destOffset = 0;

Calendar value = new GregorianCalendar();
value.setTimeZone(TimeZone.getTimeZone("UTC"));
int dateTimeLength = 7;  //yyMdhms
writer.writeUtc(value, dateTimeLength);

The ION UTC field can contain a date and time encoded using an ION specific byte encoding. The encoding varies in length depending on how much information you want to include. If you look at the example above, the last parameter to the writeUtc() method specifies the "length" of the date-time format to use. The example uses a length of 7 bytes. In the table below you can see what lengths are valid, and what date-time format they result in:

Length Description
2 Two bytes representing a year between 0 and 65535. No more information.
3 As length 2 with a byte appended representing the month of the year (from 1 to 12).
4 As length 3 with a byte appended representing the day of month (from 1 to 31).
5 As length 4 with a byte appended the hour of day (from 0 to 23).
6 As length 5 with a byte appended representing minutes (from 0 to 59).
7 As length 6 with a byte appended representing seconds (from 0 to 59 - 60 when leap seconds are introduced).
9 As length 7 with two bytes appended representing the milliseconds (0 to 999).
10 As length 7 with three bytes appended representing the microseconds (0 to 999,999).
11 As length 7 with four bytes bytes appended representing the nanoseconds (0 to 999,999,999).

Regarding the milliseconds, microseconds and nanoseconds, the date-time format can only contain one of these informations. You cannot both send milliseconds and microseconds and nanoseconds (or any combinations thereof) in an ION UTC field. However, you can derive microseconds from nanoseconds, and millseconds from microseconds. That should be enough.

As you can see, the ION UTC field encoding is similar to a standard ISO date-time format except the ION UTC encoding uses binary numbers (bytes) instead of textual numbers (characters) to represent the numbers in a date.

Also, a UTC date must always be sent in UTC time. No local date / time is allowed, and thus no time zone. Conversion to and from local date and time can be done easily at either end. Thus, the Calendar passed to the writeUtc() method must be in UTC time before calling writeUtc().

This compact encoding makes ION UTC fields take around 50% the number of bytes used by the corresponding ISO date time textual format. A bit less, actually because the ISO date-time format has a T character between date and time, and uses either Z or 5 characters to encode the time zone.

In addition to being more compact than the ISO date-time format, ION UTC fields are also faster to encode / decode. Binary numbers are faster to decode than textual numbers - especially when they consist of less bytes.

writeObjectBegin() + writeObjectEnd()

The writeObjectBegin() and writeObjectEnd() methods are used to write the beginning and "end" of an ION Object field. Actually, an ION object field has no "end" marker. The writeObjectEnd() method actually jumps to the beginning of the ION Object field and updates its length bytes (after the lead byte).

Here is an example of how to use writeObjectBegin() and writeObjectEnd():

byte[] dest = new byte[10 * 1024];
IonWriter writer = new IonWriter(dest, 0);

int objectStartIndex = writer.destIndex;

int lengthLength = 2;
writer.writeObjectBegin(lengthLength);

writer.writeKey("field0");
writer.writeInt64(123);

writer.writeKey("field1");
writer.writeInt64(456);

int objectBodyLength = writer.destIndex - (objectStartIndex + 1 + lengthLength);

writer.writeObjectEnd(objectStartIndex, lengthLength, objectBodyLength);

Notice how you have to pass a lengthLength parameter to writeObjectBegin(). This parameter tells how many bytes to reserve to write the length of the object. The length of the object is filled into these reserved length bytes in the later call to writeObjectEnd().

The example reserves 2 bytes to represent the length because we know that the length of the Object field body will never be more than 2^16 - 1 (65,535) bytes long. Actually, in this case reserving a single length byte would have been enough, as the length of the body of the Object field will not be longer than 255 bytes.

Notice also the writeKey() callS. Writing an ION Key field into an ION Object signals a key name or property name. The ION field following the Key field is the value for that key or property.

writeTableBegin() + writeTableEnd()

The writeTableBegin() and writeTableEnd() methods work similarly to the writeObjectBegin() and writeObjectEnd() methods. The writeTableBegin() method writes the beginning of an ION Table field, and the writeTableEnd() method jumps up and updates the reserved length bytes of the Table field.

Here is an example of how to use the writeTableBegin() and writeTableEnd() methods:

byte[] dest = new byte[10 * 1024];
IonWriter writer = new IonWriter(dest, 0);

int tableStartIndex = writer.destIndex;
int lengthLength = 2;
writer.writeTableBegin(lengthLength);

writer.writeKey("field0");
writer.writeKey("field1");

writer.writeInt64(123);
writer.writeInt64(456);

writer.writeInt64(111);
writer.writeInt64(222);

int tableBodyLength = writer.destIndex - (tableStartIndex + 1 + lengthLength);

writer.writeTableEnd(tableStartIndex, lengthLength, tableBodyLength);

Notice how you have to pass a lengthLength parameter to writeTableBegin(). This parameter tells how many bytes to reserve to write the length of the Table field. The length of the Table is filled into these reserved length bytes in the later call to writeTableEnd().

The example reserves 2 bytes to represent the length because we know that the length of the Table field body will never be more than 2^16 - 1 (65,535) bytes long. Actually, in this case reserving a single length byte would have been enough, as the length of the body of the Table field will not be longer than 255 bytes.

Notice also one significant difference to writing ION Object fields. The ION Table field only contains the keys for the columns once. Just like the header row of a CSV file. The example then writes 4 value fields in total (2 Int64 and 2 Float). These value fields are associated with the headers according to the sequence both are written in. The first value field is associated with the first Key field, the second value field with the second Key etc.

When you reach a value field sequence number which is greater than the number of Key fields, you start re-matching from the first Key field again (Key index goes back to 0), and matching continues up from there again. Thus, the third value field matches the first Key field, and the fourth value field matches the second Key field.

You should make sure that there are always enough value fields to match a full row of Key fields. If your Table has 4 Keys you should have a multiple of 4 value fields (0, 4, 8, 12 etc.).

Note: We are still considering if ION Table fields should contain the number of rows in the Table as one of the first elements in the Table, like is the case with Arrays. This might actually end up being the case. If we make that slight change, then the above code example will be changed a little bit. You can get an idea about how by looking at how ION Arrays are written.

writeArrayBegin() + writeArrayEnd()

The static writeArrayBegin() and writeArrayEnd() writes the beginning and end of an ION Array field. Actually, an ION Array does not have an "end" marker. Instead, writeArrayEnd() jumps up to the beginning of the Array and writes the length (in bytes) of the array body into the reserved length bytes.

Here is an example of how to write an ION Array field:

byte[] dest = new byte[10 * 1024];
IonWriter writer = new IonWriter(dest, 0);

int arrayStartIndex = writer.destIndex;
int lengthLength = 2;
writer.writeArrayBegin(lengthLength);
writer.writeInt64(3); //element count

writer.writeInt64(456);
writer.writeInt64(111);
writer.writeInt64(222);

int arrayBodyLength = writer.destIndex - (arrayStartIndex + 1 + lengthLength);

writer.writeArrayEnd(arrayStartIndex, lengthLength, arrayBodyLength);

Notice that an Array field has no keys. An ION Array is intended to be used for "anonymous" sequences of ION fields of the same type. Yes, you can actually mix the field types written into an ION Array, but this is not how the ION Array field is intended to be used. If you want a sequence of mixed field types use an ION Object without Key fields in (yes that is allowed).

Notice also how the first ION field written into the Array is an Int64-Positive which contains the number of elements in the array. For arrays smaller than 16 elements it is also allowed to use a Tiny field instead of an Int64-Positive field.

If you do not know how many elements will be in the array ahead of time, reserve a number of bytes to hold the Array length later (1 lead byte + N bytes to hold element count). This is similar to how the byte length of an Array is filled into the reserved length bytes later. You can always reserve the same number of bytes as you reserve to hold the byte length of an Array, as an Array can never have more elements than bytes inside it (the smallest ION field is 1 byte long).

IonWriter Static Methods

The IonWriter's static methods mirror the IonWriter's instance methods. The static methods are covered in the following sections.

writeBytes()

The static writeBytes() method writes a sequence (array) of raw bytes as a Bytes ION field to the given byte array. Here is an IonWriter writeBytes() example:

byte[] src  = new byte[128];

byte[] dest = new byte[10 * 1024];
int destOffset = 0;

int bytesWritten = IonWriter.writeBytes(dest, destOffset, src);

The writeBytes() method returns the total number of bytes written to the byte array, including the Bytes ION field lead byte, length bytes and the raw bytes from the source array.

writeBoolean()

The static writeBoolean() method writes a boolean ION field (encoded as a Tiny) into the given byte array at the given offset. Here is an IonWriter writeBoolean() example:

byte[] dest = new byte[10 * 1024];
int destOffset = 0;

int bytesWritten = IonWriter.writeBoolean(dest, destOffset, true);

A boolean is encoded as a Tiny ION field. A Tiny field always takes up 1 byte in total.

writeInt64()

The static method writeInt64() writes an integer up to 64 bits long into the given byte array. The ION field used will be either Int64-Positive or Int64-Negative, depending on whether the given value is positive or negative. Here is an IonWriter writeInt64() example:

byte[] dest = new byte[10 * 1024];
int destOffset = 0;

int value = 123456;
int bytesWritten = IonWriter.writeInt64(dest, destOffset, value);

The writeInt64() method returns the total number of bytes written to the byte array. An Int64-Positive / Int64-Negative ION field can be up to 64 bits (8 bytes) long, plus one extra byte for the lead field. Thus, between 2 and 9 bytes in total.

writeFloat32()

The static writeFloat32() method writes a 32 bit floating point (a Java float) as an ION Float field to the given byte array. Here is an IonWriter writeFloat32() example:

byte[] dest = new byte[10 * 1024];
int destOffset = 0;

float value = 12.34f;
int bytesWritten = IonWriter.writeFloat32(dest, destOffset, value);

A Java float is always written as an ION Float field of 4 bytes (32 bits) plus 1 lead byte, meaning 5 bytes in total.

writeFloat64()

The static writeFloat64() method writes a 64 bit floating point (Java double) to the given byte array. Here is an IonWriter writeFloat64() example:

byte[] dest = new byte[10 * 1024];
int destOffset = 0;

double value = 1234.5678d;
int bytesWritten = IonWriter.writeFloat64(dest, destOffset, value);

A Java double is always written as an ION Float field of 8 bytes (64 bits) plus 1 lead byte, meaning 9 bytes in total.

writeUtf8()

The static writeUtf8() method writes a Java String to an ION UTF-8 field. Here is an IonWriter writeUtf8() example:

byte[] dest = new byte[10 * 1024];
int destOffset = 0;

String value = "A String";
int bytesWritten = IonWriter.writeUtf8(dest, destOffset, value);

The IonWriter also contains another version of writeUtf8() which takes a byte array as parameter instead of a String. The bytes in the byte array must already be encoded as UTF-8. The bytes will not be touched by the writeUtf8() method. They are merely copied to the destination byte array. Here is an example of using that writeUtf8() method:

byte[] dest = new byte[10 * 1024];
int destOffset = 0;

byte[] value = "A String".getBytes("UTF-8");
int bytesWritten = IonWriter.writeUtf8(dest, destOffset, value);

Strings are converted to UTF-8 encoded bytes before being written to the destination byte array. If the total number of UTF-8 encoded bytes is 15 or less, they are written as an ION UTF-8-Short field. If the total number of UTF-8 encoded bytes is 16 or more, they are written as an ION UTF-8 field.

writeUtc()

The static writeUtc() method writes a Java Calendar as an ION UTC field to the given byte array. Here is a IonWriter writeUtc() example:

byte[] dest = new byte[10 * 1024];
int destOffset = 0;

Calendar value = new GregorianCalendar();
value.setTimeZone(TimeZone.getTimeZone("UTC"));
int dateTimeLength = 7;
int bytesWritten = IonWriter.writeUtc(dest, destOffset, value, dateTimeLength);

The ION UTC field can contain a date and time encoded using an ION specific byte encoding. The encoding varies in length depending on how much information you want to include. If you look at the example above, the last parameter to the writeUtc() method specifies the "length" of the date-time format to use. The example uses a length of 7 bytes. In the table below you can see what lengths are valid, and what date-time format they result in:

Length Description
2 Two bytes representing a year between 0 and 65535. No more information.
3 As length 2 with a byte appended representing the month of the year (from 1 to 12).
4 As length 3 with a byte appended representing the day of month (from 1 to 31).
5 As length 4 with a byte appended the hour of day (from 0 to 23).
6 As length 5 with a byte appended representing minutes (from 0 to 59).
7 As length 6 with a byte appended representing seconds (from 0 to 59 - 60 when leap seconds are introduced).
9 As length 7 with two bytes appended representing the milliseconds (0 to 999).
10 As length 7 with three bytes appended representing the microseconds (0 to 999,999).
11 As length 7 with four bytes bytes appended representing the nanoseconds (0 to 999,999,999).

Regarding the milliseconds, microseconds and nanoseconds, the date-time format can only contain one of these informations. You cannot both send milliseconds and microseconds and nanoseconds (or any combinations thereof) in an ION UTC field. However, you can derive microseconds from nanoseconds, and millseconds from microseconds. That should be enough.

As you can see, the ION UTC field encoding is similar to a standard ISO date-time format except the ION UTC encoding uses binary numbers (bytes) instead of textual numbers (characters) to represent the numbers in a date.

Also, a UTC date must always be sent in UTC time. No local date / time is allowed, and thus no time zone. Conversion to and from local date and time can be done easily at either end. Thus, the Calendar passed to the writeUtc() method must be in UTC time before calling writeUtc().

This compact encoding makes ION UTC fields take around 50% the number of bytes used by the corresponding ISO date time textual format. A bit less, actually because the ISO date-time format has a T character between date and time, and uses either Z or 5 characters to encode the time zone.

In addition to being more compact than the ISO date-time format, ION UTC fields are also faster to encode / decode. Binary numbers are faster to decode than textual numbers - especially when they consist of less bytes.

Writing Multiple Fields

When writing multiple ION fields after each other to the same byte array, you need to use the index of the first byte after the last written ION field as the start index for the next ION field. Here is a Java example showing you how to do that:

byte[] dest = new byte[10 * 1024];
int destOffset = 0;

int index = 0;
index += IonWriter.writeInt64  (dest, destOffset, 123456);
index += IonWriter.writeFloat32(dest, destOffset, 123.456f);
index += IonWriter.writeFloat64(dest, destOffset, 987.654d);

Writing Complex Fields

Complex ION fields are fields which can contain other fields inside them. The three most commonly used complex ION fields are Object, Table and Array. Actually, you could nest ION fields inside a Bytes field too, as the Bytes field can hold all kinds of raw bytes including ION fields.

All ION fields contain their byte length at the beginning of the field. A challenge with complex fields is that you often do not know ahead of time how many bytes the fields nested inside the complex field will take. In such situations you cannot write the number of bytes in its body at the beginning of the complex field.

To solve this problem we have a small trick. Instead of writing the number of bytes its body takes into the beginning of the complex field, we simply reserve a number of bytes big enough to hold the length. When we are done writing fields into the body of the complex field we know how many bytes these fields amount to. We can then jump back up and write the length into the reserved length bytes.

How many length bytes should you reserve? If you know that the body of a complex field will never be more than 255 bytes long, reserve 1 length byte. If the body will never be more than 65,535 bytes long, reserve 2 bytes. If the body will never be more than 16,777,216 bytes long, reserve 3 bytes etc. ION fields can have up to 15 length bytes, but you will most often never need more than 4.

The number of bytes used to represent the length (in bytes) of an ION field is often referred to as "lengthLength". This means the "length" (in bytes) needed to represent the "length" (in bytes) of the ION field body. If the value (body) length of an ION field is less than 256 bytes, then a lengthLength of 1 (byte) will suffice. If the value length of an ION field is less than 65,536 bytes, then a lengthLength of 2 (bytes) will suffice etc.

The write methods for the complex fields will help you both reserve and later fill in the lengthLength bytes.

writeObjectBegin() + writeObjectEnd()

The static writeObjectBegin() and writeObjectEnd() methods are used to write the beginning and "end" of an ION Object field. Actually, an ION object field has no "end" marker. The writeObjectEnd() method actually jumps to the beginning of the ION Object field and updates its length bytes (after the lead byte).

Here is an example of how to use writeObjectBegin() and writeObjectEnd():

byte[] dest = new byte[10 * 1024];
int destOffset = 0;

int index = 0;

int objectStartIndex = index;
int lengthLength = 2;

index += IonWriter.writeObjectBegin(dest, index, lengthLength);

index += IonWriter.writeKey(dest, index, "field0");
index += IonWriter.writeInt64  (dest, destOffset, 123456);

index += IonWriter.writeKey(dest, index, "field1");
index += IonWriter.writeFloat32(dest, destOffset, 123.456f);

// index - objectStartIndex = full byte length of object

int objectFullByteLength = index - objectStartIndex;

// Length of the value (body) of an object should not include
// the ION field lead byte (1 byte) or the length bytes (lengthLength number of bytes)

int objectBodyByteLength = objectFullByteLength - 1 - lengthLength;

IonWriter.writeObjectEnd(dest, objectStartIndex, lengthLength, objectBodyByteLength);

Notice how you have to pass a lengthLength parameter to writeObjectBegin(). This parameter tells how many bytes to reserve to write the length of the object. The length of the object is filled into these reserved length bytes in the later call to writeObjectEnd().

The example reserves 2 bytes to represent the length because we know that the length of the Object field body will never be more than 2^16 - 1 (65,535) bytes long. Actually, in this case reserving a single length byte would have been enough, as the length of the body of the Object field will not be longer than 255 bytes.

Notice also the writeKey() calls. Writing an ION Key field into an ION Object signals a key name or property name. The ION field following the Key field is the value for that key or property.

writeTableBegin() + writeTableEnd()

The static writeTableBegin() and writeTableEnd() methods work similarly to the writeObjectBegin() and writeObjectEnd() methods. The writeTableBegin() method writes the beginning of an ION Table field, and the writeTableEnd() method jumps up and updates the reserved length bytes of the Table field.

Here is an example of how to use the writeTableBegin() and writeTableEnd() methods:

byte[] dest = new byte[10 * 1024];
int destOffset = 0;

int index = 0;

int tableStartIndex = index;
int lengthLength = 2;

index += IonWriter.writeTableBegin(dest, index, lengthLength);

index += IonWriter.writeKey(dest, index, "field0");
index += IonWriter.writeKey(dest, index, "field1");

index += IonWriter.writeInt64  (dest, destOffset, 123456);
index += IonWriter.writeFloat32(dest, destOffset, 123.456f);

index += IonWriter.writeInt64  (dest, destOffset, 123456);
index += IonWriter.writeFloat32(dest, destOffset, 123.456f);

// index - tableStartIndex = full byte length of Table field

int tableFullByteLength = index - tableStartIndex;

// Length of the value (body) of an Table should not include
// the ION field lead byte (1 byte) or the length bytes (lengthLength number of bytes)

int tableBodyByteLength = tableFullByteLength - 1 - lengthLength;

IonWriter.writeTableEnd(dest, tableStartIndex, lengthLength, tableBodyByteLength);

Notice how you have to pass a lengthLength parameter to writeTableBegin(). This parameter tells how many bytes to reserve to write the length of the Table field. The length of the Table is filled into these reserved length bytes in the later call to writeTableEnd().

The example reserves 2 bytes to represent the length because we know that the length of the Table field body will never be more than 2^16 - 1 (65,535) bytes long. Actually, in this case reserving a single length byte would have been enough, as the length of the body of the Table field will not be longer than 255 bytes.

Notice also one significant difference to writing ION Object fields. The ION Table field only contains the keys for the columns once. Just like the header row of a CSV file. The example then writes 4 value fields in total (2 Int64 and 2 Float). These value fields are associated with the headers according to the sequence both are written in. The first value field is associated with the first Key field, the second value field with the second Key etc.

When you reach a value field sequence number which is greater than the number of Key fields, you start re-matching from the first Key field again (Key index goes back to 0), and matching continues up from there again. Thus, the third value field matches the first Key field, and the fourth value field matches the second Key field.

You should make sure that there are always enough value fields to match a full row of Key fields. If your Table has 4 Keys you should have a multiple of 4 value fields (0, 4, 8, 12 etc.).

Note: We are still considering if ION Table fields should contain the number of rows in the Table as one of the first elements in the Table, like is the case with Arrays. This might actually end up being the case. If we make that slight change, then the above code example will be changed a little bit. You can get an idea about how by looking at how ION Arrays are written.

writeArrayBegin() + writeArrayEnd()

The static writeArrayBegin() and writeArrayEnd() writes the beginning and end of an ION Array field. Actually, an ION Array does not have an "end" marker. Instead, writeArrayEnd() jumps up to the beginning of the Array and writes the length (in bytes) of the array body into the reserved length bytes.

Here is an example of how to write an ION Array field:

byte[] dest = new byte[10 * 1024];
int destOffset = 0;

int index = 0;

int arrayStartIndex = index;
int lengthLength = 2;

index += IonWriter.writeArrayBegin(dest, index, lengthLength);

// element count obligatory, and must come before element fields.
int arrayElementCount = 3;
index += IonWriter.writeInt64  (dest, destOffset, arrayElementCount);

// array elements.
index += IonWriter.writeInt64  (dest, destOffset, 123);
index += IonWriter.writeInt64  (dest, destOffset, 456);
index += IonWriter.writeInt64  (dest, destOffset, 789);

// index - arrayStartIndex = full byte length of Table field,
int arrayFullByteLength = index - arrayStartIndex;

// Length of the value (body) of an Array should not include
// the ION field lead byte (1 byte) or the length bytes (lengthLength number of bytes)
int arrayBodyByteLength = arrayFullByteLength - 1 - lengthLength;

IonWriter.writeArrayEnd(dest, arrayStartIndex, lengthLength, arrayBodyByteLength);

Notice that an Array field has no keys. An ION Array is intended to be used for "anonymous" sequences of ION fields of the same type. Yes, you can actually mix the field types written into an ION Array, but this is not how the ION Array field is intended to be used. If you want a sequence of mixed field types use an ION Object without Key fields in (yes that is allowed).

Notice also how the first ION field written into the Array is an Int64-Positive which contains the number of elements in the array. For arrays smaller than 16 elements it is also allowed to use a Tiny field instead of an Int64-Positive field.

If you do not know how many elements will be in the array ahead of time, reserve a number of bytes to hold the Array length later (1 lead byte + N bytes to hold element count). This is similar to how the byte length of an Array is filled into the reserved length bytes later. You can always reserve the same number of bytes as you reserve to hold the byte length of an Array, as an Array can never have more elements than bytes inside it (the smallest ION field is 1 byte long).

Jakob Jenkov




Copyright  Jenkov Aps
Close TOC