-
Notifications
You must be signed in to change notification settings - Fork 120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Multiasic]: Provide namespace support for ipNetToMediaPhysAddress #129
[Multiasic]: Provide namespace support for ipNetToMediaPhysAddress #129
Conversation
MIB in rfc1213. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
|
||
def update_data(self): | ||
self.arp_dest_map = {} | ||
self.arp_dest_list = [] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No need if you immediately assign them. #Closed
dest_map, dest_list = self.get_arp_table() | ||
self.arp_dest_map.update(dest_map) | ||
self.arp_dest_list.extend(dest_list) | ||
# Revert back to original namespace |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Revert back to original namespace [](start = 16, length = 35)
Could you revert back only at the end of loop? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Modified implementation to use NEIGH_TABLE instead of using linux namespace.
for ns in self.ns_list: | ||
dest_map = {} | ||
dest_list = [] | ||
ns_fd = open(mibs.get_netns_path(nsname = ns)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ns_fd = open(mibs.get_netns_path(nsname = ns)) [](start = 16, length = 46)
use with
statement, no need to close
explicitly #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
vModified implementation to use NEIGH_TABLE instead of using linux namespace.
self.arp_dest_map.update(dest_map) | ||
self.arp_dest_list.extend(dest_list) | ||
# Revert back to original namespace | ||
mibs.setns(current_fd, mibs.CLONE_NEWNET) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mibs.setns(current_fd, mibs.CLONE_NEWNET) [](start = 16, length = 41)
Use a finally
to make sure it is executed even there is exception? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Modified implementation to use NEIGH_TABLE instead of using linux namespace.
src/sonic_ax_impl/mibs/__init__.py
Outdated
@@ -8,6 +10,10 @@ | |||
from ax_interface.util import oid2tuple | |||
from sonic_ax_impl import logger | |||
|
|||
libc = cdll.LoadLibrary('libc.so.6') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
libc [](start = 0, length = 4)
handle exception or error? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed, not used anymore.
src/sonic_ax_impl/mibs/__init__.py
Outdated
|
||
redis_kwargs = {'unix_socket_path': '/var/run/redis/redis.sock'} | ||
|
||
def setns(fd, nstype): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
setns [](start = 4, length = 5)
These new features has nothing to do with mibs
, and it may be reused outside this repo? Could you move into a common repo? Better to encapsulate a class. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed this.
src/sonic_ax_impl/mibs/__init__.py
Outdated
|
||
redis_kwargs = {'unix_socket_path': '/var/run/redis/redis.sock'} | ||
|
||
def setns(fd, nstype): | ||
if hasattr(fd, 'fileno'): | ||
fd = fd.fileno() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why we need the flexibility? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed this function, not using this.
src/sonic_ax_impl/mibs/__init__.py
Outdated
_setns(fd, nstype) | ||
|
||
def get_netns_path(nspath=None, nsname=None, nspid=None): | ||
if nsname: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nsname [](start = 7, length = 6)
check it is a string. #Closed
src/sonic_ax_impl/mibs/__init__.py
Outdated
|
||
def get_netns_path(nspath=None, nsname=None, nspid=None): | ||
if nsname: | ||
if (os.path.isdir(NET_NS_PATH) == False): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
os.path.isdir(NET_NS_PATH) [](start = 12, length = 26)
We only need to check this path once? #Closed
src/sonic_ax_impl/mibs/__init__.py
Outdated
fd = fd.fileno() | ||
_setns(fd, nstype) | ||
|
||
def get_netns_path(nspath=None, nsname=None, nspid=None): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nspath=None [](start = 19, length = 11)
Can you remove it from arg list? #Closed
src/sonic_ax_impl/mibs/__init__.py
Outdated
fd = fd.fileno() | ||
_setns(fd, nstype) | ||
|
||
def get_netns_path(nspath=None, nsname=None, nspid=None): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nsname=None, nspid=None [](start = 32, length = 23)
Better to separate into 2 functions. #Closed
src/sonic_ax_impl/mibs/__init__.py
Outdated
|
||
def get_netns_path(nspath=None, nsname=None, nspid=None): | ||
if nsname: | ||
if (os.path.isdir(NET_NS_PATH) == False): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if (os.path.isdir(NET_NS_PATH) == False): | |
if not os.path.isdir(NET_NS_PATH): | |
``` #Closed |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed this function, not used.
src/sonic_ax_impl/mibs/__init__.py
Outdated
if ns_list is None: | ||
return None | ||
else: | ||
return ns_list |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just return ns_list
#Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed this, not used.
src/sonic_ax_impl/mibs/__init__.py
Outdated
nspath = '/proc/%d/ns/net' % nspid | ||
return nspath | ||
|
||
def get_namespace_list(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
get_namespace_list [](start = 4, length = 18)
Add some unit tests #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added unit-tests
arp_dest_list.append(subid) | ||
return arp_dest_map, arp_dest_list | ||
|
||
def update_data(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
update_data [](start = 8, length = 11)
Add some unit tests #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added unit-tests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As comments
why not use the neighbor table information in the app db? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please switch to use neighbor table in app db.
sure, will update the changes. |
Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
namespaces. Add function to retrieve data from NEIGH_TABLE in APP_DB of each namespace. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
src/sonic_ax_impl/mibs/__init__.py
Outdated
@@ -62,6 +62,13 @@ | |||
|
|||
redis_kwargs = {'unix_socket_path': '/var/run/redis/redis.sock'} | |||
|
|||
|
|||
def get_neigh_info(neigh_key): | |||
neigh_key = neigh_key[neigh_key.find(':')+1:len(neigh_key)] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
':' [](start = 41, length = 3)
Instead of hard-coded, could you get it by get_separator()
. But only call this function minimal times.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed this, to only split neigh_key information of IPv4 neigh address.
def reinit_data(self): | ||
if len(self.db_conn) > 1: | ||
Namespace.connect_all_dbs(self.db_conn, mibs.APPL_DB) | ||
self.neigh_key_list = Namespace.dbs_keys_namespace(self.db_conn, mibs.APPL_DB, "NEIGH_TABLE:*") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
dbs_keys_namespace [](start = 44, length = 18)
Function does not exist. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Requires PR#137
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added function as PR #137 is not merged yet.
neigh_str = neigh_key.decode() | ||
db_index = self.neigh_key_list[neigh_key] | ||
dev, ip = mibs.get_neigh_info(neigh_str) | ||
""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
""" [](start = 12, length = 3)
Docstrings are for all public modules, functions, classes, and methods. Here '# ' will be better. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
""" | ||
if dev == "eth0": | ||
continue | ||
neigh_info = self.db_conn[db_index].get_all(mibs.APPL_DB, neigh_key, blocking=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
get_all [](start = 48, length = 7)
Did you call get_all
too many times? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for all NEIGH_TABLE keys, we have to call get_all
tests/test_arp.py
Outdated
@@ -22,6 +22,7 @@ | |||
class TestSonicMIB(TestCase): | |||
@classmethod | |||
def setUpClass(cls): | |||
tests.mock_tables.python_arptable.arp_filename = '/arp.txt' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
arp_filename [](start = 42, length = 12)
Not used? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is used in mock python_arptable.
For single namespace, we will use arp.txt - this will have mgmt interfaces and front panel interfaces data.
For multiple namespace, we use host_arp.txt - this have only mgmt interface related data.
Rest of namespace information will be retrieved from mock appl_db.json.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is to test single ASIC, and we don't need to mock kernel ARP any more?
In reply to: 451193110 [](ancestors = 451193110)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, we don't need this, fixed.
@@ -0,0 +1,46 @@ | |||
IP address HW type Flags HW address Mask Device |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IP address [](start = 0, length = 10)
Delete original tests/mock_tables/arp.txt file? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done.
@@ -86,5 +86,17 @@ | |||
"INTF_TABLE:PortChannel01:10.10.0.1/31": { | |||
"scope": "global", | |||
"family": "IPv4" | |||
}, | |||
"NEIGH_TABLE:Ethernet0:10.0.0.0": { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
10.0.0.0 [](start = 25, length = 8)
Add some mock data for IPv6
Add some mock data for PortChannel
Will new mock data need more test cases? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added data for IPv6 and Port Channel. Added one more test case to make sure that eth0 of namespace, which is a part of docker0 bridge, is not used, for multi-asic platform.
tests/namespace/test_arp.py
Outdated
# for updater in self.lut.updater_instances: | ||
# updater.update_data() | ||
# updater.reinit_data() | ||
# updater.update_data() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it working?
Or remove it? #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is not there in the latest file.
d728e97
from kernel and namespace arp table from NEIGH_TABLE in APP_DB | ||
in each namespace. | ||
""" | ||
self._update_host_data() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
_update_host_data [](start = 13, length = 17)
For single ASIC, we should treat 'NEIGH_TABLE' as the ground truth. No need to check arp_table.
And we need to adjust test for single ASIC. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated to read 'NEIGH_TABLE' for single asic as well. Updated mock db to reflect this change.
Changes made so that for single asic platform, arp table information is retrieved from NEIGH_TABLE. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
mock arp.txt file as data will be retrieved from APPL_DB. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
namespace or docker's eth0 information is not retrieved. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
as information is retrieved from NEIGH_TABLE. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
for neigh_key in self.neigh_key_list: | ||
neigh_str = neigh_key.decode() | ||
db_index = self.neigh_key_list[neigh_key] | ||
neigh_info = self.db_conn[db_index].get_all(mibs.APPL_DB, neigh_key, blocking=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
blocking=True [](start = 81, length = 13)
Let's use non blocking, which make snmpagent robust even if there is an issue in any namespace, such as process crash, DB corruption. #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated as per review comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Non blocking may return a None (please verify), then you need to handle this corner case.
In reply to: 458323712 [](ancestors = 458323712)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added check, will continue with other keys if None is returned on get_all on any key.
to make sure snmpagent is robust even if there is issue in any one namespace. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
for entry in python_arptable.get_arp_table(): | ||
dev = entry['Device'] | ||
mac = entry['HW address'] | ||
ip = entry['IP address'] | ||
self._update_arp_info(dev, mac, ip) | ||
|
||
def _update_namespace_data(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
_update_namespace_data [](start = 8, length = 22)
_update_from_db #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated.
Namespace.connect_all_dbs(self.db_conn, mibs.APPL_DB) | ||
self.neigh_key_list = Namespace.dbs_keys_namespace(self.db_conn, mibs.APPL_DB, "NEIGH_TABLE:*") | ||
|
||
def _update_host_data(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
_update_host_data [](start = 8, length = 17)
_update_from_arptable #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated.
Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
This pull request introduces 1 alert when merging 70633d6 into c702a57 - view on LGTM.com new alerts:
|
Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
I think this description is not up to date. |
Updated description. |
) * [Multiasic]: Provide namespace support for ipNetToMediaPhysAddress MIB in rfc1213. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com> * [Namespace][ArpUpdater]: Update ArpUpdater to support multiple namespaces. Add function to retrieve data from NEIGH_TABLE in APP_DB of each namespace. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com> * Updated as per review comments. Changes made so that for single asic platform, arp table information is retrieved from NEIGH_TABLE. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>
MIB in rfc1213.
Signed-off-by: SuvarnaMeenakshi sumeenak@microsoft.com
- What I did
Provide namespace support for ipNetToMediaPhysAddress MIB.
- How I did it
In multi-asic platform, retrieve host arp information from kernel using python_arptable.
Retrieve namespace arp information from NEIGH_TABLE in APP_DB.
For single-asic platform - Implementation modified to read NEIGH_TABLE in APP_DB.
- How to verify it
Tested on single-asic platform to make sure the output is the same as before the changes were added.
Tested on multi-asic platform.
- Description for the changelog