The ION APIs in Grid Ops are designed to work with byte arrays. We find it easier to work with byte arrays than with streams for the following reasons:
- A byte array can be read multiple times and passed easily along between components internally in memory.
- Byte arrays tend to work well with CPU caches.
- Byte arrays are fast to read and write, and provide arbitrary access. Streams usually only provide one-way iteration.
- Byte arrays are easy to write to both files and network connections. You can write an ION structure to a byte array once, and then write the same byte array to both to a log and over the network.
If you are working on a one-thread-per-connection type platform (like traditional Java EE), working with arrays might be more painful than if you work on a single-threaded, reactive platform like Vert.x or Node.JS. However, we believe the single-threaded platforms to be the future - which is another reason to choose to have Grid Ops work on byte arrays.
ION Readers and Writers
The ION tools in Grid Ops consists of two ION readers and two ION writers.
The first pair of ION reader + writer just writes the primitive ION fields directly to a byte array, and reads ION fields from a byte array. This reader + writer pair is the fastest to use. If you need maximum performance or flexibility, this is the reader / writer pair you should use. You can read more about these here:
The second pair of ION reader + writer reads and writes Java objects to and from byte arrays. This reader + writer pair uses Java Reflection to read and write the fields of the objects from and to binary ION data. The reflection based reader + writer are not as fast as the raw reader + writer pair, but much easier to use. You can read more about the reflection based ION APis here:
It is also possible to combine the two sets of readers and writers. For instance, you can iterate through a large ION data set with the raw reader, and then read individual objects out from it using the reflection based object reader. Or, switch between writing raw data manually and writing Java objects using the reflection based writer.
The ION APIs are not yet perfectly geared towards this mixing on the surface, but the underlying implementations were made with this in mind, so it is mostly a matter of exposing this functionality in the public APIs. This will be improved over time.