Skip to content

Commit

Permalink
Fix missing dataclass attributes info
Browse files Browse the repository at this point in the history 
Summary:
The team working on [Hydra](https://hydra.cc/docs/intro/), a framework that enables storing configs in yaml files that correspond to Python schemas, heavily depends on glean to improve the dev experience. They pointed out that some dataclass fields were not indexed correctly.

Example, in the class MastTorchXOptions, the fields `data_project` and `model_type_name` don't appear in the [Public Fields list](https://www.internalfb.com/code/symbol/fbsource/py/fbcode/gen_ai.genie.components.launchers.types.MastTorchXOptions)

https://www.internalfb.com/code/fbsource/[24a0922d3b7d]/fbcode/gen_ai/genie/components/launchers/types.py?lines=120-121%2C175-176%2C178-179

Those symbol do exist, but they don't have any scope info. See [data_project](https://www.internalfb.com/code/symbol/fbsource/py/fbcode/gen_ai.genie.components.launchers.types.MastTorchXOptions.data_project) vs [cluster_name](https://www.internalfb.com/code/symbol/fbsource/py/fbcode/gen_ai.genie.components.launchers.types.MastTorchXOptions.cluster_name) symbol pages

 {F1916669718}

The problem is that we don't construct the fact `VariableDefinition` for those attributes which has the container information
https://www.internalfb.com/code/fbsource/[2a9473826736]/fbcode/glean/schema/source/python.angle?lines=103-109

This is happening because we are explicitly excluding attributes without a default value in our indexer here:

https://www.internalfb.com/code/fbsource/[d4a3490eeee9]/fbcode/glean/lang/python/indexers/core.py?lines=1113-1114

Reviewed By: iamirzhan

Differential Revision: D64041090

fbshipit-source-id: b9e5883933c0dc0d60eb0a7e1d3001f32ce73cbf
  • Loading branch information
@facebook-github-bot
Rubmary Rojas authored and facebook-github-bot committed Oct 9, 2024
1 parent e310725 commit 2751dc9
Show file tree
Hide file tree
Showing 11 changed files with 188 additions and 11 deletions.
59 changes: 58 additions & 1 deletion glean/glass/test/regression/tests/python/relatedNeighborhood.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
{
"childrenContained": [
"test/py/lib.py/lib.HelperClass.another_method",
"test/py/lib.py/lib.HelperClass.method"
"test/py/lib.py/lib.HelperClass.method",
"test/py/lib.py/lib.HelperClass.value"
],
"childrenExtended": [],
"containsParents": [
Expand Down Expand Up @@ -242,6 +243,62 @@
}
],
"visibility": 20
},
"test/py/lib.py/lib.HelperClass.value": {
"contains_relation": {
"firstChild": "nondeterministic",
"firstParent": "nondeterministic",
"hasMoreChildren": false,
"hasMoreParents": false
},
"extends_relation": {
"firstChild": "nondeterministic",
"firstParent": "nondeterministic",
"hasMoreChildren": false,
"hasMoreParents": false
},
"kind": 10,
"language": 7,
"location": {
"filepath": "lib.py",
"range": {
"columnBegin": 5,
"columnEnd": 56,
"lineBegin": 10,
"lineEnd": 10
},
"repository": "test"
},
"modifiers": [],
"name": {
"container": "lib.HelperClass",
"localName": "value"
},
"pretty_comments": [],
"repo_hash": "testhash",
"signature": "value: TypeVar(\"THelperClass\", bound=\"HelperClass\")",
"sym": "test/py/lib.py/lib.HelperClass.value",
"sym_location": {
"filepath": "lib.py",
"range": {
"columnBegin": 5,
"columnEnd": 56,
"lineBegin": 10,
"lineEnd": 10
},
"repository": "test"
},
"sym_other_locations": [],
"type_xrefs": [
{
"span": {
"length": 5,
"start": 0
},
"type": "test/py/lib.py/lib.HelperClass.value"
}
],
"visibility": 20
}
}
}
Expand Down
7 changes: 7 additions & 0 deletions glean/glass/test/regression/tests/python/searchSymbol_prefix_empty_name.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -705,6 +705,13 @@
"symbol": "test/py/lib.py/lib.HelperClass.method"
},
{
"context": {
"qname": {
"container": "lib",
"localName": "HelperClass"
},
"symbol": "test/py/lib.py/lib.HelperClass"
},
"kind": 10,
"language": 7,
"location": {
Expand Down
18 changes: 17 additions & 1 deletion glean/glass/test/regression/tests/python/searchSymbol_prefix_empty_name_describe.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1481,6 +1481,7 @@
},
"pretty_comments": [],
"repo_hash": "testhash",
"signature": "value: TypeVar(\"THelperClass\", bound=\"HelperClass\")",
"sym": "test/py/lib.py/lib.HelperClass.value",
"sym_location": {
"filepath": "lib.py",
Expand All @@ -1493,7 +1494,15 @@
"repository": "test"
},
"sym_other_locations": [],
"type_xrefs": [],
"type_xrefs": [
{
"span": {
"length": 5,
"start": 0
},
"type": "test/py/lib.py/lib.HelperClass.value"
}
],
"visibility": 20
},
{
Expand Down Expand Up @@ -3271,6 +3280,13 @@
"symbol": "test/py/lib.py/lib.HelperClass.method"
},
{
"context": {
"qname": {
"container": "lib",
"localName": "HelperClass"
},
"symbol": "test/py/lib.py/lib.HelperClass"
},
"kind": 10,
"language": 7,
"location": {
Expand Down
7 changes: 7 additions & 0 deletions glean/glass/test/regression/tests/python/searchSymbol_prefix_empty_name_ignorecase.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -705,6 +705,13 @@
"symbol": "test/py/lib.py/lib.HelperClass.method"
},
{
"context": {
"qname": {
"container": "lib",
"localName": "HelperClass"
},
"symbol": "test/py/lib.py/lib.HelperClass"
},
"kind": 10,
"language": 7,
"location": {
Expand Down
18 changes: 18 additions & 0 deletions glean/lang/codemarkup/tests/python/cases/xrefs/contains_child_entity.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,24 @@
}
}
},
{
"key": {
"parent": {
"python": {
"decl": { "cls": { "key": { "name": { "key": "lib.HelperClass" } } } }
}
},
"child": {
"python": {
"decl": {
"variable": {
"key": { "name": { "key": "lib.HelperClass.instance_var" } }
}
}
}
}
}
},
{
"key": {
"parent": {
Expand Down
2 changes: 1 addition & 1 deletion glean/lang/codemarkup/tests/python/cases/xrefs/contains_child_entity.perf
View file
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{ "@generated": null, "python.Contains.4": 29 }
{ "@generated": null, "python.Contains.4": 30 }
4 changes: 2 additions & 2 deletions glean/lang/codemarkup/tests/python/cases/xrefs/entity_modifiers.perf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@
"python.ClassDefinition.4": 7,
"python.DeclarationLocation.4": 51,
"python.FunctionDefinition.4": 26,
"python.Name.4": 1,
"python.VariableDefinition.4": 1
"python.Name.4": 2,
"python.VariableDefinition.4": 2
}
36 changes: 36 additions & 0 deletions glean/lang/codemarkup/tests/python/cases/xrefs/search_related_contains_child_of_parent.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,42 @@
"style": 0
}
},
{
"key": {
"query": 2,
"parent": {
"parent": {
"python": {
"decl": {
"cls": { "key": { "name": { "key": "lib.HelperClass" } } }
}
}
},
"location": {
"name": "lib.HelperClass",
"file": { "key": "lib.py" },
"location": { "span": { "start": 351, "length": 451 } }
}
},
"child": {
"child": {
"python": {
"decl": {
"variable": {
"key": { "name": { "key": "lib.HelperClass.instance_var" } }
}
}
}
},
"location": {
"name": "lib.HelperClass.instance_var",
"file": { "key": "lib.py" },
"location": { "span": { "start": 440, "length": 17 } }
}
},
"style": 0
}
},
{
"key": {
"query": 2,
Expand Down
6 changes: 3 additions & 3 deletions glean/lang/codemarkup/tests/python/cases/xrefs/search_related_contains_child_of_parent.perf
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"@generated": null,
"python.Contains.4": 29,
"python.DeclarationDefinition.4": 54,
"python.DeclarationLocation.4": 54
"python.Contains.4": 30,
"python.DeclarationDefinition.4": 56,
"python.DeclarationLocation.4": 56
}
36 changes: 36 additions & 0 deletions glean/lang/codemarkup/tests/python/cases/xrefs/search_related_contains_parent_of_child.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,42 @@
"style": 0
}
},
{
"key": {
"query": 3,
"parent": {
"parent": {
"python": {
"decl": {
"cls": { "key": { "name": { "key": "lib.HelperClass" } } }
}
}
},
"location": {
"name": "lib.HelperClass",
"file": { "key": "lib.py" },
"location": { "span": { "start": 351, "length": 451 } }
}
},
"child": {
"child": {
"python": {
"decl": {
"variable": {
"key": { "name": { "key": "lib.HelperClass.instance_var" } }
}
}
}
},
"location": {
"name": "lib.HelperClass.instance_var",
"file": { "key": "lib.py" },
"location": { "span": { "start": 440, "length": 17 } }
}
},
"style": 0
}
},
{
"key": {
"query": 3,
Expand Down
6 changes: 3 additions & 3 deletions glean/lang/codemarkup/tests/python/cases/xrefs/search_related_contains_parent_of_child.perf
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
{
"@generated": null,
"python.ClassDefinition.4": 7,
"python.DeclarationDefinition.4": 58,
"python.DeclarationLocation.4": 58,
"python.DeclarationDefinition.4": 60,
"python.DeclarationLocation.4": 60,
"python.FunctionDefinition.4": 13,
"python.Module.4": 8,
"python.NameToSName.4": 6,
"python.SNameToName.4": 2,
"python.VariableDefinition.4": 7
"python.VariableDefinition.4": 8
}

0 comments on commit 2751dc9

Please sign in to comment.