Building Two-Level Graph#
Two-level graph is a way to express business-logic once and provide it on-demand.
Expressions in Hiku is a DSL, which is used to expose data from a low-level graph (internal) into a high-level graph (public).
Prerequisites#
Note
Source code of this example can be found on GitHub.
In order to show this feature we will try to adapt our
previous example, actor
table was removed and image
table was added:
from sqlalchemy import create_engine, MetaData, Table, Column
from sqlalchemy import Integer, Unicode, ForeignKey, select
metadata = MetaData()
image_table = Table(
'image',
metadata,
Column('id', Integer, primary_key=True),
Column('name', Unicode, nullable=False),
)
character_table = Table(
'character',
metadata,
Column('id', Integer, primary_key=True),
Column('image_id', ForeignKey('image.id')),
Column('name', Unicode),
)
sa_engine = create_engine('sqlite://')
metadata.create_all(sa_engine)
sa_engine.execute(image_table.insert().values([
dict(id=1, name='j.kirk.jpg'),
dict(id=2, name='spock.jpg'),
dict(id=3, name='l.mccoy.jpg'),
]))
sa_engine.execute(character_table.insert().values([
dict(id=1, image_id=1, name='James T. Kirk'),
dict(id=2, image_id=2, name='Spock'),
dict(id=3, image_id=3, name='Leonard McCoy'),
]))
Low-level graph definition#
Low-level graph is a graph, which exposes all our data sources, database for example. So this graph definition wouldn’t be much different from our previous graph definition:
1from hiku.graph import Graph, Root, Node, Field, Link
2from hiku.types import TypeRef, Sequence, Optional
3from hiku.engine import pass_context, Nothing
4from hiku.sources.sqlalchemy import FieldsQuery
5
6SA_ENGINE_KEY = 'sa-engine'
7
8image_query = FieldsQuery(SA_ENGINE_KEY, image_table)
9
10character_query = FieldsQuery(SA_ENGINE_KEY, character_table)
11
12def direct_link(ids):
13 return ids
14
15def maybe_direct_link(ids):
16 return [id_ if id_ is not None else Nothing
17 for id_ in ids]
18
19@pass_context
20def to_characters_query(ctx):
21 query = select([character_table.c.id])
22 return [row.id for row in ctx[SA_ENGINE_KEY].execute(query)]
23
24_GRAPH = Graph([
25 Node('Image', [
26 Field('id', None, image_query),
27 Field('name', None, image_query),
28 ]),
29 Node('Character', [
30 Field('id', None, character_query),
31 Field('image_id', None, character_query),
32 Field('name', None, character_query),
33 Link('image', Optional[TypeRef['Image']],
34 maybe_direct_link, requires='image_id'),
35 ]),
36 Root([
37 Link('characters', Sequence[TypeRef['Character']],
38 to_characters_query, requires=None),
39 ]),
40])
This example shows a Link
[33-34] with
Optional
type. This is because column
character.image_id
can be equal to null
.
Optional
type requires to use
Nothing
constant in the maybe_direct_link
[15] function in order to indicate that there is nothing to link to. This
special constant is used instead of None
, because None
can be a valid
value.
For testing purposes let’s define helper function execute
:
from hiku.engine import Engine
from hiku.result import denormalize
from hiku.readers.graphql import read
from hiku.executors.sync import SyncExecutor
hiku_engine = Engine(SyncExecutor())
def execute(graph, query_string):
query = read(query_string)
result = hiku_engine.execute(graph, query, {SA_ENGINE_KEY: sa_engine})
return denormalize(graph, result)
So let’s query some data, needed to show characters with their photos:
test_low_level():
result = execute(_GRAPH, '{ characters { name image { id name } } }')
assert result == {
'characters': [
{'name': 'James T. Kirk',
'image': {'id': 1, 'name': 'j.kirk.jpg'}},
{'name': 'Spock',
'image': {'id': 2, 'name': 'spock.jpg'}},
{'name': 'Leonard McCoy',
'image': {'id': 3, 'name': 'l.mccoy.jpg'}},
],
}
What’s wrong with this query?
{ characters { name image { id name } } }
Result of this query doesn’t give us ready to use data representation - we have
to compute image url in order to show this information. Additionally, we have to
remember, that we should include query fragment { image { id name } }
in
every query, when we need to construct url for this image.
High-level graph definition#
So our goal is to get rid of this implementation details and to be able to make queries like this:
{ characters { name image-url } }
Instead of explicitly loading image data { image { id name } }
, we want to
load ready-to-use image url. All we need is an ability to store information
about image-url
field computation and which data it needs for it’s
computation.
And here is when and why we need to implement two-level graph. Low-level graph exposes all of our data sources. High-level graph is used to express our business-logic based on low-level graph, and hides it’s implementation details.
1
2from hiku.types import Record, Integer, String
3from hiku.expr.core import S, define, if_some
4from hiku.sources.graph import SubGraph
5
6@define(Record[{'id': Integer, 'name': String}])
7def image_url(image):
8 return 'http://example.com/{id}-{name}'.format(id=image['id'],
9 name=image['name'])
10
11character_sg = SubGraph(_GRAPH, 'Character')
12
13GRAPH = Graph([
14 Node('Character', [
15 Field('id', None, character_sg),
16 Field('name', None, character_sg),
17 Field('image-url', None, character_sg.c(
18 if_some([S.img, S.this.image],
19 image_url(S.img),
20 'http://example.com/no-photo.jpg'),
21 )),
22 ]),
23 Root([
24 Link('characters', Sequence[TypeRef['Character']],
25 to_characters_query, requires=None),
26 ]),
Hold on, as there are lots of new things in the high-level graph definition above. Those are expressions.
Expressions#
In order to compose queries to low-level graph, we need to use expressions. Expressions are used to compute high-level fields from low-level graph.
SubGraph#
SubGraph
source [10] is used to refer to
low-level node. Low-level node and it’s high-level counterpart are basically
refer to the same logical entity (Character) and they share same unique
identifiers (Character.id
), used to identify every instance of the entity.
SubGraph
is used along with
Expr
fields to define expressions, which
represent how to compute high-level representation of data from low-level graph.
S
factory object#
hiku.expr.core.S
- is a special factory object, used to create
symbols on the fly. S.foo
means just foo
. It exists only because Python
doesn’t support unbound symbols.
S.this
#
S.this
is a special case, it refers to the low-level counterpart of the
current node. So S.this.name
[15] is a name
field of the
Character
node from the low-level graph. As you can see, to expose low-level
fields in the high-level node without modification, you just need to refer them
using symbols with their names.
Expression functions#
In order to make data modifications, we will need to use more complex expressions. Hiku already has several built-in functions:
each()
if_()
if_some()
@define decorator#
You are able to use your custom functions.
define()
decorator should be used to make them suitable for use in the Hiku’s expressions.
As you can see, we defined image_url
function [6] to compute image
url, and we declared argument types, which this function should accept, using
define()
decorator. Here @define(Record[...])
[7]
means that decorated function accepts one argument (only positional arguments
are supported), which should be a record with at least two fields – id
and
name
, and these fields should be with specified types:
Integer
and String
.
Type system and type checking plays a big role here. Expressions are declarative
and Hiku can analyze them in order to know which data from the low-level graph
should be loaded to compute high-level fields. When you request some fields in a
query to high-level graph ({ image-url }
), Hiku will automatically generate
query for low-level graph ({ image { id name } }
).
In our example above we can also see consequences of using type checking – need
to use if_some()
function [17] before passing
character’s image into image_url
function [6]. It is used because
S.this.image
is of type Optional[Record[...]]
and it can’t be passed
directly to the image_url
function, which requires non-optional
Record[...]
type. if_some()
function will unpack
optional type into regular type (bound to the symbol S.img
[17])
and then we can freely use it in the “then” clause of the
if_some()
expression [18] and be sure that it’s
value wouldn’t be equal to None
. In the “else” clause we will return url
of the standard “no-photo” image [19]. Without using
if_some()
function Hiku will raise type error.
Testing our high-level graph:
test_high_level():
result = execute(GRAPH, '{ characters { name image-url } }')
assert result == {
'characters': [
{'name': 'James T. Kirk',
'image-url': 'http://example.com/1-j.kirk.jpg'},
{'name': 'Spock',
'image-url': 'http://example.com/2-spock.jpg'},
{'name': 'Leonard McCoy',
'image-url': 'http://example.com/3-l.mccoy.jpg'},
],
As you can see, the goal is achieved.