Using Protocol Buffers#
Query format#
It is possible to serialize queries using Protocol Buffers in binary form, instead of sending it as a text in edn or GraphQL format.
Hiku has a hiku/protobuf/query.proto file, which describes message types for query serialization.
Here is how they can be used to build query:
from hiku.protobuf import query_pb2
node = query_pb2.Node()
link = node.items.add().link
link.name = 'characters'
field = link.node.items.add().field
field.name = 'name'
This query is equivalent to this query in edn format:
[{:characters [:name]}]
And is equivalent to this query in GraphQL format:
{
characters {
name
}
}
Note
Protocol Buffers has it’s own ability to specify requirements for
get operations – google.protobuf.FieldMask
, but this approach has
limitations for our use. For example, field options can’t be expressed with
field masks. So it would be hard to utilize field masks for our use case
and that’s why Hiku provides it’s own message types for queries.
Query export#
In Python it is not required to use example above in order to build query using
Protocol Buffers message classes. Hiku provides handy
export()
function to transform query into
Protocol Buffers message:
from hiku.builder import build, Q
from hiku.export.protobuf import export
query = build([
Q.characters[
Q.name,
],
])
message = export(query)
assert message == node
binary_message = message.SerializeToString()
Query reading#
In order to execute query, Hiku provides read()
function, which can be used to deserialize query from Protocol Buffers message:
from hiku.readers.protobuf import read
query = read(binary_message)
result = hiku_engine.execute(graph, query)
Result serialization#
The main advantage of using Hiku with Protocol Buffers is to give efficient binary format for result serialization, which can be safely read in any other language with Protocol Buffers support.
Note
Hiku is only suitable with latest 3rd version of Protocol Buffers format. This is because only in 3rd version of Protocol Buffers all message fields are strictly optional, and this opens possibility for clients to express their requirements and server will return only what client cares about.
Here is our example of a graph, similar to the one from Basics:
GRAPH = Graph([
Node('Character', [
Field('name', String, character_data),
Field('species', String, character_data),
]),
Root([
Link('characters', Sequence[TypeRef['Character']],
to_characters_link, requires=None),
]),
])
This graph can be expressed like this in example.proto
file:
syntax = "proto3";
message Character {
string name = 1;
string species = 2;
}
message Root {
repeated Character characters = 1;
}
Now we can generate example_pb2.py
file from example.proto
by using
protoc
compiler. And this is how it can be used to serialize result:
In this example server will compute only name
field of the Character
type, because this is the only field, which was specified in the query.
Note
Using Protocol Buffers “as is” still not the most efficient way to
send result back to the client. Result is sent in denormalized form, so it
contains duplicates of the same data. More efficient way would be more
complicated and probably will be implemented in the future. See
Netflix/Falcor and Om Next as examples of using normalized results.
See also how Hiku stores result internally: hiku.result