Querying Amazon Neptune with Gremlin

This note shows how to query Amazon Neptune graph data using Gremlin.

Takahiro Iwasa

2021-11-27

7 min read

Amazon Neptune is a managed graph database supporting Gremlin and SPARQL. The examples below focus on Gremlin traversals.

What is a Graph Database

Graph databases are optimized for storing and querying relationships between entities. Unlike relational or NoSQL databases, graph databases excel in scenarios where relationships are complex and performance is critical.

In graph databases:

• Vertices represent entities.
• Edges define relationships between vertices.
• Both vertices and edges can have properties (key-value pairs), which is called a property graph.

Transactions in Neptune

Neptune manages read-only and mutation queries using distinct transaction isolation levels. Refer to the official documentation.

For read-only queries, Neptune uses snapshot isolation within MultiVersion Concurrency Control (MVCC). This ensures that dirty reads, non-repeatable reads, and phantom reads are prevented by using a snapshot taken at the start of the transaction.

For mutation queries, Neptune uses READ COMMITTED isolation to prevent dirty reads. Additionally, Neptune locks the record set being read, ensuring that non-repeatable reads and phantom reads are avoided.

Traversal Examples

Preparing Graph Data

This example uses a graph where vertices have an age property and edges define a weight property.

The sample data above can be loaded with the commands below. %%gremlin is a Jupyter Notebook magic command specifically designed for use with Neptune Workbench.

1
%%gremlin
2
// Drop existing data
3
g.V().drop()

1
%%gremlin
2
// Add Vertices
3
g.addV('person').property(id, 'A').property('age', 30)
4
 .addV('person').property(id, 'B').property('age', 25)
5
 .addV('person').property(id, 'C').property('age', 35)
6
 .addV('person').property(id, 'D').property('age', 20)
7
 .addV('person').property(id, 'E').property('age', 18)
8
 .addV('person').property(id, 'F').property('age', 25)
9
 .addV('person').property(id, 'G').property('age', 10)
10
 .addV('person').property(id, 'H').property('age', 15)

1
%%gremlin
2
// Add Edges
3
g.V('A').addE('know').to(g.V('B')).property('weight', 1.0)
4
 .V('A').addE('know').to(g.V('C')).property('weight', 0.9)
5
 .V('A').addE('know').to(g.V('H')).property('weight', 0.5)
6
 .V('B').addE('know').to(g.V('D')).property('weight', 0.8)
7
 .V('B').addE('know').to(g.V('E')).property('weight', 0.4)
8
 .V('C').addE('know').to(g.V('F')).property('weight', 0.5)
9
 .V('C').addE('know').to(g.V('G')).property('weight', 0.6)
10
 .V('D').addE('know').to(g.V('E')).property('weight', 0.7)
11
 .V('H').addE('know').to(g.V('E')).property('weight', 1.0)
12
 .V('H').addE('know').to(g.V('G')).property('weight', 1.0)

Example 1: Retrieving All Vertices

1
%%gremlin
2
// Extract Vertices
3
g.V()
4
 .project('id', 'properties') // Projection
5
 .by(id()).by(valueMap()) // valueMap returns properties of vertices.

Result:

Example 2: Traversing Connections

Retrieve all persons older than 25 and connected to 'A' within two levels:

1
%%gremlin
2
// Extract persons (entities) which are older than 25 years old and connected from A up to 2nd.
3
g.V('A')
4
 .repeat(outE().inV()).times(2).emit() // Repeat traversal of adjacent vertices twice
5
 .has('age', gte(25)) // Greater than or equal 25 years old
6
 .project('id', 'age')
7
 .by(id()).by(values('age'))

Result:

Example 3: Filtering by Weight

Find persons where the multiplied weight exceeds 0.5:

1
%%gremlin
2
// Start traversal at A which extracts vertices that have a multiplied weight value greater than 0.5
3
g.withSack(1.0f).V('A') // Sack can be used to store temporary data
4
 // Multiply a weight value of an out-directed edge by a sack value, and traverse all in-directed vertices
5
 .repeat(outE().sack(mult).by('weight').inV().simplePath()).emit()
6
 .where(sack().is(gt(0.5))) // A sack value greater than 0.5
7
 .dedup() // deduplication
8
 .project('id', 'weight')
9
 .by(id).by(sack())

Result:

Visualizing Graphs

Neptune Workbench offers tools to visualize graphs interactively. Refer to the official documentation for more details.

Visualizing the graph uses a similar traversal, with display options added:

1
%%gremlin -d T.id -de weight
2
// -d specifies the vertex property to display
3
// -de specifies the edge property to display
4

5
// Execute traversal from example 3
6
g.withSack(1.0f).V('A') // Sack is used to store temporary data
7
 .repeat(outE().sack(mult).by('weight').inV().simplePath()).emit() // Traverse with edge weight
8
 .where(sack().is(gt(0.5))) // Filter paths where the sack value > 0.5
9
 .dedup() // Remove duplicate paths
10
 .path() // Extract path data
11
 .by(elementMap()) // Display properties of vertices and edges

Example Output:

Once executed, navigate to the Graph tab in Neptune Workbench to visualize the result. The interface supports drag, zoom-in, and zoom-out operations, allowing for intuitive graph exploration.

Appendix 1: Tree Structures in Relational Databases (RDB)

Tree structures can be modeled in relational databases (RDB) using various approaches. In some scenarios, graph databases may not be necessary.

However, it is important to note that the Naive Tree model is often suboptimal for many cases, as discussed in “SQL Antipatterns”.

• Naive Tree
  • Expression: t1.id = t2.parent_id
  • Pros
    • Simple implementation (Adjacency List)
  • Cons
    • Difficult to extract non-adjacent nodes
    • Complex SQL
    • Low performance
• Path Enumeration
  • Expression: path LIKE '1/2%'
  • Pros
    • Simplifies extracting non-adjacent nodes
  • Cons
    • Complex INSERT/UPDATE/DELETE operations
    • Limited by column max length
• Nested Sets
  • Expression: Left > 1 AND Right < 6
  • Pros
    • Efficient for querying non-adjacent nodes
  • Cons
    • Complex INSERT/UPDATE operations
    • Limited by column max length
    • Non-intuitive structure
• Closure Table
  • Expression: Separate table for the tree
  • Pros
    • Efficient for querying all nodes
    • Handles INSERT/UPDATE/DELETE easily
  • Cons
    • Data can grow significantly in size
    • INSERT/UPDATE/DELETE can have low performance

When storing data in an RDB, querying relationships can become increasingly complex. SQL queries often grow in complexity, and representing interconnected data within a row-column structure is unintuitive.

Appendix 2: Transactions

Dirty Read

When Tx1 updates the row, Tx2 reads the row, and Tx1 fails or rollbacks, Tx2 reads data which has not been reflected.

Non-repeatable Read

When Tx1 reads the row, Tx2 updates or deletes the row and commits, and Tx1 reads the row again, Tx1 reads committed data different from the previous one.

Phantom Read

When Tx1 reads the records, Tx2 inserts or deletes some rows and commits, and Tx1 reads the rows again, Tx1 reads the rows different from the previous ones.

Isolation Levels

Conclusion

Loading a small graph into Neptune and querying it with Gremlin traversals showed how multi-hop relationship queries and transaction isolation work in a graph database. repeat(outE().inV()).times(2).emit() in example 2 walks two hops out from vertex A in a single expression, and withSack(1.0f) in example 3 carries a running weight along each path as the traversal proceeds — both read close to plain English despite doing what would otherwise require recursive CTEs or several chained self-joins in SQL. That gap only widens with the RDB tree-modeling comparison in the appendix: adjacency lists get expensive to query past one level deep, and alternatives like closure tables solve it by trading write complexity and storage growth for read performance, whereas Neptune’s snapshot isolation on reads and locked, READ COMMITTED semantics on writes handle the concurrency question without that tradeoff. For data where the relationships themselves are the thing being queried, that combination is what makes a graph schema worth reaching for over a relational schema.

About the author

Takahiro Iwasa

LinkedIn

GitHub

Software Developer

This blog shares technical notes from hands-on projects—architecture, implementation, and AWS service integrations.