Add org permissions fetcher

CHANGES.md

@@ -1,3 +1,9 @@
+# [0.11.0](https://github.com/ComplianceAsCode/auditree-arboretum/releases/tag/v0.11.0)
+
+- [ADDED] Github org integrity fetcher added to `permissions`.
+- [ADDED] Github org integrity forks fetcher functionality added Github org integrity fetcher.
+- [CHANGED] Github org integrity collaborators fetcher functionality added to Github org integrity permissions fetcher.
+
 # [0.10.0](https://github.com/ComplianceAsCode/auditree-arboretum/releases/tag/v0.10.0)
 
 - [ADDED] Organization repository direct collaborators check added to `permissions`.

arboretum/__init__.py

@@ -14,4 +14,4 @@
 # limitations under the License.
 """Arboretum - Checking your compliance & security posture, continuously."""
 
-__version__ = '0.10.0'
+__version__ = '0.11.0'

arboretum/permissions/README.md

@@ -16,14 +16,11 @@ how to include the fetchers and checks from this library in your downstream proj
 
 ## Fetchers
 
-### Organization Integrity (Repository Collaborators)
+### Organization Integrity Permissions
 
-* Class: [GithubOrgCollaboratorsFetcher][gh-org-fetcher]
-* Purpose: Writes the details of collaborators in Github organizations to the evidence locker. This fetcher class is only meant for use with Github or Github Enterprise organizations.
-* Behavior: For each Github organization specified, an evidence file per collaborator type (affiliation) is stored in
-the locker containing collaborator details for the specified repositories in the organization. The default is to
-retrieve all collaborators by affiliation from all repositories in each specified Github organization.  TTL is set to 1
-day.
+* Class: [GithubOrgPermissionFetcher][fetch-org-permissions]
+* Purpose: Writes the details of collaborators and repository forks in Github organizations to the evidence locker. This fetcher class is only meant for use with Github or Github Enterprise organizations.
+* Behavior: For each Github organization specified, Github collaborator and Github fork evidence files per collaborator type (affiliation) are stored in the locker containing details for the specified repositories in the organization. The default is to retrieve all collaborators and all forks by affiliation from all repositories in each specified Github organization.  TTL is set to 1 day.
 * Configuration elements:
   * `org.permissions.org_integrity.orgs`
      * Required
@@ -36,7 +33,7 @@ day.
            * Optional
            * List of strings in the form of `["my-repo", "my-other-repo"]`.
            * Defaults to all repositories in the organization.
-           * Use if looking to filter collaborator evidence to a subset of repositories in the organization otherwise do not include.
+           * Use if looking to filter permissions evidence to a subset of repositories in the organization otherwise do not include.
         * `collaborator_types`
            * Optional
            * List of strings in the form of `["all", "direct", "outside"]`.
@@ -92,7 +89,7 @@ day.
 * Import statement:
 
   ```python
-  from arboretum.permissions.fetchers.github.fetch_org_collaborators import GithubOrgCollaboratorsFetcher
+  from arboretum.permissions.fetchers.github.fetch_org_permissions import GithubOrgPermissionFetcher
   ```
 
 ## Checks
@@ -164,6 +161,6 @@ direct collaborators matching the exceptions are found.
 [auditree-framework]: https://github.com/ComplianceAsCode/auditree-framework
 [auditree-framework documentation]: https://complianceascode.github.io/auditree-framework/
 [usage]: https://github.com/ComplianceAsCode/auditree-arboretum#usage
-[gh-org-fetcher]: https://github.com/ComplianceAsCode/auditree-arboretum/blob/main/arboretum/permissions/fetchers/github/fetch_org_collaborators.py
+[fetch-org-permissions]: https://github.com/ComplianceAsCode/auditree-arboretum/blob/main/arboretum/permissions/fetchers/github/fetch_org_permissions.py
 [repository-permissions]: https://docs.github.com/en/free-pro-team@latest/github/setting-up-and-managing-organizations-and-teams/repository-permission-levels-for-an-organization
-[org-collaborators-check]: https://github.com/ComplianceAsCode/auditree-arboretum/blob/main/arboretum/permissions/checks/test_org_collaborators.py
+[org-collaborators-check]: https://github.com/ComplianceAsCode/auditree-arboretum/blob/main/arboretum/permissions/checks/test_org_collaborators.py

arboretum/permissions/fetchers/github/fetch_org_permissions.py

@@ -0,0 +1,52 @@
+# -*- mode:python; coding:utf-8 -*-
+# Copyright (c) 2021 IBM Corp. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Github organization permissions fetcher."""
+import json
+
+import arboretum.permissions.fetchers.github.fetch_org_collaborators as collabs
+
+
+class GithubOrgPermissionsFetcher(collabs.GithubOrgCollaboratorsFetcher):
+    """Fetch Github permissions evidence."""
+
+    def fetch_repo_forks(self):
+        """Fetch Github repository forks."""
+        for config in self.config.get('org.permissions.org_integrity.orgs'):
+            host, org = config['url'].rsplit('/', 1)
+            url_hash = collabs.get_sha256_hash([config['url']], 10)
+            path = ['permissions', f'gh_forks_{url_hash}.json']
+            description = f'Forks of repos in the {org} GH org'
+            self.config.add_evidences(
+                [
+                    collabs.RawEvidence(
+                        path[1], path[0], collabs.DAY, description
+                    )
+                ]
+            )
+            with collabs.raw_evidence(self.locker, '/'.join(path)) as evidence:
+                if evidence:
+                    if host not in self.gh_pool:
+                        self.gh_pool[host] = collabs.Github(base_url=host)
+                    if not config.get('repos'):
+                        repos = self.gh_pool[host].paginate_api(
+                            f'orgs/{org}/repos'
+                        )
+                        config['repos'] = [repo['name'] for repo in repos]
+                    forks = {}
+                    for repo in config['repos']:
+                        forks[repo] = self.gh_pool[host].paginate_api(
+                            f'repos/{org}/{repo}/forks'
+                        )
+                    evidence.set_content(json.dumps(forks))