Support bi-directional graph traversal command `graphlookup` #5113

qianheng-aws · 2026-02-05T02:34:41Z

Description

[Describe what this change achieves]

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
New PPL command checklist all confirmed.
API changes companion pull request created.
Commits are signed per the DCO using --signoff or -s.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Lantao Jin <ltjin@amazon.com>

…graphlookup # Conflicts: # core/src/main/java/org/opensearch/sql/analysis/Analyzer.java # core/src/main/java/org/opensearch/sql/ast/AbstractNodeVisitor.java # core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java # ppl/src/main/antlr/OpenSearchPPLParser.g4

Signed-off-by: Heng Qian <qianheng@amazon.com>

coderabbitai · 2026-02-05T02:34:49Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch poc/poc_graphlookup

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

LantaoJin · 2026-02-05T03:03:54Z

Change the target branch to feature/graphlookup

Signed-off-by: Heng Qian <qianheng@amazon.com>

LantaoJin · 2026-02-06T06:45:15Z

opensearch/src/main/java/org/opensearch/sql/opensearch/functions/GraphLookupBfsFunction.java

+ *
+ * <p>Returns: List - array of [row_fields..., depth?]
+ */
+public class GraphLookupBfsFunction extends ImplementorUDF {


Could the classes GraphLookupBfsFunction and GraphLookupFunction prune?

LantaoJin · 2026-02-06T07:03:20Z

...h/src/main/java/org/opensearch/sql/opensearch/storage/scan/CalciteEnumerableGraphLookup.java

+          }
+        }
+
+        if (++currentLevelDepth > graphLookup.maxDepth) break;


Seems it processes one additional level. For example maxDepth=2, it processes 3 levels total instead of stopping at level 2:
1st level done, then check (++0 > 2 ❎), continue;
2nd level done, then check (++1 > 2 ❎), continue;
3rd level done, then check (++ 2 > 2 ✅), stop

That's by design. It starts from 0 hop which stands for direct connection. It means we can set maxDepth=0. You can refer to mongdb's definition about this param, it should be a non-negative integer

LantaoJin · 2026-02-06T07:04:41Z

...h/src/main/java/org/opensearch/sql/opensearch/storage/scan/CalciteEnumerableGraphLookup.java

+            if (graphLookup.depthField != null) {
+              Object[] rowWithDepth = new Object[rowArray.length + 1];
+              System.arraycopy(rowArray, 0, rowWithDepth, 0, rowArray.length);
+              rowWithDepth[rowArray.length] = currentLevelDepth;


currentLevelDepth represents the depth of SOURCE nodes being queried, not the TARGET nodes being added to results. Target nodes should be at depth currentLevelDepth + 1?

Tests with depthField use default maxDepth=0, which only processes one iteration, so all results correctly show depth=0. Tests with maxDepth > 0 don't verify depth values.

LantaoJin · 2026-02-06T07:07:07Z

...h/src/main/java/org/opensearch/sql/opensearch/storage/scan/CalciteEnumerableGraphLookup.java

+    }
+
+    @Override
+    public Object current() {


Lacks ResourceMonitor health checks during BFS traversal. Need to add periodic health check monitor.isHealthy() in moveNext() and performBfs().

The monitor is inner both scan for source table and lookup table. Since our all operators actually share the same jvm and monitor. It will be triggered in each iteration inner scan of source or lookup table.

LantaoJin · 2026-02-06T07:12:37Z

...h/src/main/java/org/opensearch/sql/opensearch/storage/scan/CalciteEnumerableGraphLookup.java

+
+        query = QueryBuilders.boolQuery().should(query).should(backQuery);
+      }
+      CalciteEnumerableIndexScan newScan = (CalciteEnumerableIndexScan) this.lookupScan.copy();


The code creates new scan instance but never close it? can you debug to check if this instance be closed expectedly.

Signed-off-by: Heng Qian <qianheng@amazon.com>

LantaoJin and others added 15 commits January 8, 2026 15:54

succeed to graph lookup single index

061061e

Signed-off-by: Lantao Jin <ltjin@amazon.com>

Implement graph lookup RelNode

8d591e1

Signed-off-by: Heng Qian <qianheng@amazon.com>

Refine - remove depth from BFS node

0f7c0ef

Signed-off-by: Heng Qian <qianheng@amazon.com>

Support bidirectional mode

bca0ac1

Signed-off-by: Heng Qian <qianheng@amazon.com>

Support anonymize graph lookup

9761001

Signed-off-by: Heng Qian <qianheng@amazon.com>

Fix UT

5c936e9

Signed-off-by: Heng Qian <qianheng@amazon.com>

Add IT

b54d1de

Signed-off-by: Heng Qian <qianheng@amazon.com>

Add limitation for GraphLookup

1bc06a8

Signed-off-by: Heng Qian <qianheng@amazon.com>

Simplify GraphLookup param names

f577aef

Signed-off-by: Heng Qian <qianheng@amazon.com>

Refine IT

61e8526

Signed-off-by: Heng Qian <qianheng@amazon.com>

Support value of list; Support retrieve circle edges also

906efc5

Signed-off-by: Heng Qian <qianheng@amazon.com>

Add documentation for graph lookup

845b450

Signed-off-by: Heng Qian <qianheng@amazon.com>

Don't include loop edges

857044e

Signed-off-by: Heng Qian <qianheng@amazon.com>

Refine code

b61e020

Signed-off-by: Heng Qian <qianheng@amazon.com>

LantaoJin changed the base branch from main to feature/graphlookup February 5, 2026 03:03

LantaoJin added the feature label Feb 5, 2026

qianheng-aws added 4 commits February 5, 2026 11:31

spotlessApply

fabbb54

Signed-off-by: Heng Qian <qianheng@amazon.com>

Refine code

84199b8

Signed-off-by: Heng Qian <qianheng@amazon.com>

Filter visited nodes in search query

fcda2df

Signed-off-by: Heng Qian <qianheng@amazon.com>

Fix UT

6430d6c

Signed-off-by: Heng Qian <qianheng@amazon.com>

LantaoJin changed the title ~~Poc/poc graphlookup~~ Support bi-directional graph traversal command graphlookup Feb 6, 2026

LantaoJin reviewed Feb 6, 2026

View reviewed changes

qianheng-aws added 5 commits February 6, 2026 15:17

Add parameter supportArray for handling fields with array values

bb9e516

Signed-off-by: Heng Qian <qianheng@amazon.com>

Remove unused code

6af4d84

Signed-off-by: Heng Qian <qianheng@amazon.com>

Support batch mode

041be84

Signed-off-by: Heng Qian <qianheng@amazon.com>

Close lookup table scan

0069b87

Signed-off-by: Heng Qian <qianheng@amazon.com>

refine code

0f68927

Signed-off-by: Heng Qian <qianheng@amazon.com>

Add param usePIT

072059e

Signed-off-by: Heng Qian <qianheng@amazon.com>

LantaoJin marked this pull request as ready for review February 10, 2026 03:10

LantaoJin requested review from GumpacG, MaxKsyunz, RyanL1997, Swiddis, YANG-DB, Yury-Fridlyand, acarbonetto, anirudha, dai-chen, derek-ho, forestmvey, joshuali925, kavithacm, mengweieric, noCharger, penghuo, ps48, seankao-az, vamsimanohar, ykmr1224 and yuancu as code owners February 10, 2026 03:10

LantaoJin merged commit 81eae25 into feature/graphlookup Feb 10, 2026
101 of 108 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support bi-directional graph traversal command `graphlookup` #5113

Support bi-directional graph traversal command `graphlookup` #5113

Uh oh!

qianheng-aws commented Feb 5, 2026

Uh oh!

coderabbitai bot commented Feb 5, 2026 •

edited

Loading

Review skipped

Uh oh!

LantaoJin commented Feb 5, 2026

Uh oh!

LantaoJin Feb 6, 2026

Uh oh!

LantaoJin Feb 6, 2026

Uh oh!

qianheng-aws Feb 6, 2026 •

edited

Loading

Uh oh!

LantaoJin Feb 6, 2026

Uh oh!

LantaoJin Feb 6, 2026

Uh oh!

qianheng-aws Feb 6, 2026

Uh oh!

LantaoJin Feb 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Support bi-directional graph traversal command graphlookup #5113

Support bi-directional graph traversal command graphlookup #5113

Uh oh!

Conversation

qianheng-aws commented Feb 5, 2026

Description

Related Issues

Check List

Uh oh!

coderabbitai bot commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

LantaoJin commented Feb 5, 2026

Uh oh!

LantaoJin Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

LantaoJin Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

qianheng-aws Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LantaoJin Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

LantaoJin Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

qianheng-aws Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

LantaoJin Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Support bi-directional graph traversal command `graphlookup` #5113

Support bi-directional graph traversal command `graphlookup` #5113

coderabbitai bot commented Feb 5, 2026 •

edited

Loading

qianheng-aws Feb 6, 2026 •

edited

Loading