Querying Amazon Neptune with Gremlin

Load property graph data into Amazon Neptune and query relationships with Gremlin traversals.

Takahiro Iwasa

2021-11-27

7 min read

Amazon Neptune is a managed graph database supporting Gremlin and SPARQL. The examples below focus on Gremlin traversals.

What Is a Graph Database?

Graph databases are optimized for storing and querying relationships between entities. They are useful when traversing interconnected data would require complex joins in a relational model.

In graph databases:

• Vertices represent entities.
• Edges define relationships between vertices.
• Both vertices and edges can have properties (key-value pairs). This model is called a property graph.

Transactions in Neptune

Neptune uses different transaction isolation levels for read-only and mutation queries. See the official documentation for details.

For read-only queries, Neptune uses snapshot isolation with MultiVersion Concurrency Control (MVCC). Each query reads from a snapshot taken at the start of the transaction, preventing dirty reads, non-repeatable reads, and phantom reads.

For mutation queries, Neptune uses READ COMMITTED isolation to prevent dirty reads. It also locks the record ranges being read to prevent non-repeatable reads and phantom reads.

Traversal Examples

Preparing Graph Data

This example uses vertices with an age property and edges with a weight property.

Load the sample data with the following commands. %%gremlin is a Jupyter Notebook magic command available in Neptune Workbench.

1
%%gremlin
2
// Drop existing data
3
g.V().drop()

1
%%gremlin
2
// Add Vertices
3
g.addV('person').property(id, 'A').property('age', 30)
4
 .addV('person').property(id, 'B').property('age', 25)
5
 .addV('person').property(id, 'C').property('age', 35)
6
 .addV('person').property(id, 'D').property('age', 20)
7
 .addV('person').property(id, 'E').property('age', 18)
8
 .addV('person').property(id, 'F').property('age', 25)
9
 .addV('person').property(id, 'G').property('age', 10)
10
 .addV('person').property(id, 'H').property('age', 15)

1
%%gremlin
2
// Add Edges
3
g.V('A').addE('know').to(g.V('B')).property('weight', 1.0)
4
 .V('A').addE('know').to(g.V('C')).property('weight', 0.9)
5
 .V('A').addE('know').to(g.V('H')).property('weight', 0.5)
6
 .V('B').addE('know').to(g.V('D')).property('weight', 0.8)
7
 .V('B').addE('know').to(g.V('E')).property('weight', 0.4)
8
 .V('C').addE('know').to(g.V('F')).property('weight', 0.5)
9
 .V('C').addE('know').to(g.V('G')).property('weight', 0.6)
10
 .V('D').addE('know').to(g.V('E')).property('weight', 0.7)
11
 .V('H').addE('know').to(g.V('E')).property('weight', 1.0)
12
 .V('H').addE('know').to(g.V('G')).property('weight', 1.0)

Example 1: Retrieving All Vertices

1
%%gremlin
2
// Extract Vertices
3
g.V()
4
 .project('id', 'properties') // Projection
5
 .by(id()).by(valueMap()) // valueMap returns properties of vertices.

Result:

Example 2: Traversing Connections

Retrieve all people aged 25 or older who are connected to 'A' within two hops:

1
%%gremlin
2
// Extract persons (entities) which are older than 25 years old and connected from A up to 2nd.
3
g.V('A')
4
 .repeat(outE().inV()).times(2).emit() // Repeat traversal of adjacent vertices twice
5
 .has('age', gte(25)) // Greater than or equal 25 years old
6
 .project('id', 'age')
7
 .by(id()).by(values('age'))

Result:

Example 3: Filtering by Weight

Find vertices whose product of edge weights from 'A' exceeds 0.5:

1
%%gremlin
2
// Start traversal at A which extracts vertices that have a multiplied weight value greater than 0.5
3
g.withSack(1.0f).V('A') // Sack can be used to store temporary data
4
 // Multiply a weight value of an out-directed edge by a sack value, and traverse all in-directed vertices
5
 .repeat(outE().sack(mult).by('weight').inV().simplePath()).emit()
6
 .where(sack().is(gt(0.5))) // A sack value greater than 0.5
7
 .dedup() // deduplication
8
 .project('id', 'weight')
9
 .by(id).by(sack())

Result:

Visualizing Graphs

Neptune Workbench can visualize query results interactively. See the official documentation for details.

Visualizing the graph uses a similar traversal, with display options added:

1
%%gremlin -d T.id -de weight
2
// -d specifies the vertex property to display
3
// -de specifies the edge property to display
4

5
// Execute traversal from example 3
6
g.withSack(1.0f).V('A') // Sack is used to store temporary data
7
 .repeat(outE().sack(mult).by('weight').inV().simplePath()).emit() // Traverse with edge weight
8
 .where(sack().is(gt(0.5))) // Filter paths where the sack value > 0.5
9
 .dedup() // Remove duplicate paths
10
 .path() // Extract path data
11
 .by(elementMap()) // Display properties of vertices and edges

Example Output:

After running the query, open the Graph tab in Neptune Workbench to visualize the result. The interface supports dragging and zooming to explore the graph.

Appendix 1: Tree Structures in Relational Databases (RDB)

Tree structures can be modeled in relational databases (RDBs) in several ways, so a graph database is not required for every hierarchical dataset.

However, the Naive Tree model has limitations, as discussed in “SQL Antipatterns”.

• Naive Tree
  • Expression: t1.id = t2.parent_id
  • Pros
    • Simple implementation (Adjacency List)
  • Cons
    • Difficult to extract non-adjacent nodes
    • Complex SQL
    • Low performance
• Path Enumeration
  • Expression: path LIKE '1/2%'
  • Pros
    • Simplifies extracting non-adjacent nodes
  • Cons
    • Complex INSERT/UPDATE/DELETE operations
    • Limited by column max length
• Nested Sets
  • Expression: Left > 1 AND Right < 6
  • Pros
    • Efficient for querying non-adjacent nodes
  • Cons
    • Complex INSERT/UPDATE operations
    • Limited by column max length
    • Non-intuitive structure
• Closure Table
  • Expression: Separate table for the tree
  • Pros
    • Efficient for querying all nodes
    • Handles INSERT/UPDATE/DELETE easily
  • Cons
    • Data can grow significantly in size
    • INSERT/UPDATE/DELETE can have low performance

In a relational model, queries become more complex as the number of relationship levels increases. Representing and traversing highly connected data with rows and columns may require recursive queries or repeated joins.

Appendix 2: Transactions

Dirty Read

Tx1 updates a row, Tx2 reads the uncommitted value, and then Tx1 fails or rolls back. Tx2 has read data that was never committed.

Non-repeatable Read

Tx1 reads a row, Tx2 updates or deletes it and commits, and then Tx1 reads it again. Tx1 sees a different committed value from its first read.

Phantom Read

Tx1 reads a set of rows, Tx2 inserts or deletes matching rows and commits, and then Tx1 repeats the query. Tx1 sees a different set of rows from its first read.

Isolation Levels

Conclusion

The examples load a small property graph into Neptune and use Gremlin to query multi-hop relationships and cumulative edge weights.

In example 2, repeat(outE().inV()).times(2).emit() traverses up to two hops from vertex A. In example 3, withSack(1.0f) carries the cumulative weight along each path.

Relational tree models involve different tradeoffs: adjacency lists are simple to update but require more complex queries for distant nodes, while closure tables improve those queries at the cost of additional storage and write processing.

When relationships are a primary part of the query, a graph model can express these traversals more directly than repeated relational joins.

About the author

Takahiro Iwasa

LinkedIn

GitHub

Software Developer

This blog shares technical notes from hands-on projects—architecture, implementation, and AWS service integrations.