Building Two-Level Graph#

Two-level graph is a way to express business-logic once and provide it on-demand.

Expressions in Hiku is a DSL, which is used to expose data from a low-level graph (internal) into a high-level graph (public).

Prerequisites#

Note

Source code of this example can be found on GitHub.

In order to show this feature we will try to adapt our previous example, actor table was removed and image table was added:

from sqlalchemy import create_engine, MetaData, Table, Column
from sqlalchemy import Integer, Unicode, ForeignKey, select

metadata = MetaData()

image_table = Table(
    'image',
    metadata,
    Column('id', Integer, primary_key=True),
    Column('name', Unicode, nullable=False),
)

character_table = Table(
    'character',
    metadata,
    Column('id', Integer, primary_key=True),
    Column('image_id', ForeignKey('image.id')),
    Column('name', Unicode),
)

sa_engine = create_engine('sqlite://')
metadata.create_all(sa_engine)

sa_engine.execute(image_table.insert().values([
    dict(id=1, name='j.kirk.jpg'),
    dict(id=2, name='spock.jpg'),
    dict(id=3, name='l.mccoy.jpg'),
]))
sa_engine.execute(character_table.insert().values([
    dict(id=1, image_id=1, name='James T. Kirk'),
    dict(id=2, image_id=2, name='Spock'),
    dict(id=3, image_id=3, name='Leonard McCoy'),
]))

Low-level graph definition#

Low-level graph is a graph, which exposes all our data sources, database for example. So this graph definition wouldn’t be much different from our previous graph definition:

 1from hiku.graph import Graph, Root, Node, Field, Link
 2from hiku.types import TypeRef, Sequence, Optional
 3from hiku.engine import pass_context, Nothing
 4from hiku.sources.sqlalchemy import FieldsQuery
 5
 6SA_ENGINE_KEY = 'sa-engine'
 7
 8image_query = FieldsQuery(SA_ENGINE_KEY, image_table)
 9
10character_query = FieldsQuery(SA_ENGINE_KEY, character_table)
11
12def direct_link(ids):
13    return ids
14
15def maybe_direct_link(ids):
16    return [id_ if id_ is not None else Nothing
17            for id_ in ids]
18
19@pass_context
20def to_characters_query(ctx):
21    query = select([character_table.c.id])
22    return [row.id for row in ctx[SA_ENGINE_KEY].execute(query)]
23
24_GRAPH = Graph([
25    Node('Image', [
26        Field('id', None, image_query),
27        Field('name', None, image_query),
28    ]),
29    Node('Character', [
30        Field('id', None, character_query),
31        Field('image_id', None, character_query),
32        Field('name', None, character_query),
33        Link('image', Optional[TypeRef['Image']],
34             maybe_direct_link, requires='image_id'),
35    ]),
36    Root([
37        Link('characters', Sequence[TypeRef['Character']],
38             to_characters_query, requires=None),
39    ]),
40])

This example shows a Link [33-34] with Optional type. This is because column character.image_id can be equal to null.

Optional type requires to use Nothing constant in the maybe_direct_link [15] function in order to indicate that there is nothing to link to. This special constant is used instead of None, because None can be a valid value.

For testing purposes let’s define helper function execute:

from hiku.engine import Engine
from hiku.result import denormalize
from hiku.readers.graphql import read
from hiku.executors.sync import SyncExecutor

hiku_engine = Engine(SyncExecutor())


def execute(graph, query_string):
    query = read(query_string)
    result = hiku_engine.execute(graph, query, {SA_ENGINE_KEY: sa_engine})
    return denormalize(graph, result)

So let’s query some data, needed to show characters with their photos:

test_low_level():
result = execute(_GRAPH, '{ characters { name image { id name } } }')
assert result == {
    'characters': [
        {'name': 'James T. Kirk',
         'image': {'id': 1, 'name': 'j.kirk.jpg'}},
        {'name': 'Spock',
         'image': {'id': 2, 'name': 'spock.jpg'}},
        {'name': 'Leonard McCoy',
         'image': {'id': 3, 'name': 'l.mccoy.jpg'}},
    ],
}

What’s wrong with this query?

{ characters { name image { id name } } }

Result of this query doesn’t give us ready to use data representation - we have to compute image url in order to show this information. Additionally, we have to remember, that we should include query fragment { image { id name } } in every query, when we need to construct url for this image.

High-level graph definition#

So our goal is to get rid of this implementation details and to be able to make queries like this:

{ characters { name image-url } }

Instead of explicitly loading image data { image { id name } }, we want to load ready-to-use image url. All we need is an ability to store information about image-url field computation and which data it needs for it’s computation.

And here is when and why we need to implement two-level graph. Low-level graph exposes all of our data sources. High-level graph is used to express our business-logic based on low-level graph, and hides it’s implementation details.

 1
 2from hiku.types import Record, Integer, String
 3from hiku.expr.core import S, define, if_some
 4from hiku.sources.graph import SubGraph
 5
 6@define(Record[{'id': Integer, 'name': String}])
 7def image_url(image):
 8    return 'http://example.com/{id}-{name}'.format(id=image['id'],
 9                                                   name=image['name'])
10
11character_sg = SubGraph(_GRAPH, 'Character')
12
13GRAPH = Graph([
14    Node('Character', [
15        Field('id', None, character_sg),
16        Field('name', None, character_sg),
17        Field('image-url', None, character_sg.c(
18            if_some([S.img, S.this.image],
19                    image_url(S.img),
20                    'http://example.com/no-photo.jpg'),
21        )),
22    ]),
23    Root([
24        Link('characters', Sequence[TypeRef['Character']],
25             to_characters_query, requires=None),
26    ]),

Hold on, as there are lots of new things in the high-level graph definition above. Those are expressions.

Expressions#

In order to compose queries to low-level graph, we need to use expressions. Expressions are used to compute high-level fields from low-level graph.

SubGraph#

SubGraph source [10] is used to refer to low-level node. Low-level node and it’s high-level counterpart are basically refer to the same logical entity (Character) and they share same unique identifiers (Character.id), used to identify every instance of the entity.

SubGraph is used along with Expr fields to define expressions, which represent how to compute high-level representation of data from low-level graph.

S factory object#

hiku.expr.core.S - is a special factory object, used to create symbols on the fly. S.foo means just foo. It exists only because Python doesn’t support unbound symbols.

S.this#

S.this is a special case, it refers to the low-level counterpart of the current node. So S.this.name [15] is a name field of the Character node from the low-level graph. As you can see, to expose low-level fields in the high-level node without modification, you just need to refer them using symbols with their names.

Expression functions#

In order to make data modifications, we will need to use more complex expressions. Hiku already has several built-in functions:

  • each()

  • if_()

  • if_some()

@define decorator#

You are able to use your custom functions. define() decorator should be used to make them suitable for use in the Hiku’s expressions.

As you can see, we defined image_url function [6] to compute image url, and we declared argument types, which this function should accept, using define() decorator. Here @define(Record[...]) [7] means that decorated function accepts one argument (only positional arguments are supported), which should be a record with at least two fields – id and name, and these fields should be with specified types: Integer and String.

Type system and type checking plays a big role here. Expressions are declarative and Hiku can analyze them in order to know which data from the low-level graph should be loaded to compute high-level fields. When you request some fields in a query to high-level graph ({ image-url }), Hiku will automatically generate query for low-level graph ({ image { id name } }).

In our example above we can also see consequences of using type checking – need to use if_some() function [17] before passing character’s image into image_url function [6]. It is used because S.this.image is of type Optional[Record[...]] and it can’t be passed directly to the image_url function, which requires non-optional Record[...] type. if_some() function will unpack optional type into regular type (bound to the symbol S.img [17]) and then we can freely use it in the “then” clause of the if_some() expression [18] and be sure that it’s value wouldn’t be equal to None. In the “else” clause we will return url of the standard “no-photo” image [19]. Without using if_some() function Hiku will raise type error.

Testing our high-level graph:

test_high_level():
result = execute(GRAPH, '{ characters { name image-url } }')
assert result == {
    'characters': [
        {'name': 'James T. Kirk',
         'image-url': 'http://example.com/1-j.kirk.jpg'},
        {'name': 'Spock',
         'image-url': 'http://example.com/2-spock.jpg'},
        {'name': 'Leonard McCoy',
         'image-url': 'http://example.com/3-l.mccoy.jpg'},
    ],

As you can see, the goal is achieved.