Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(WIP/Collaboration) rewrites for delegation graph issues #846

Closed
wants to merge 52 commits into from

Commits on Apr 3, 2019

  1. Add helper funcs for roledb: get_delegation, _is_top_level_role

    These aid in the roledb rewrite to start to address Issue #660.
    
    Also add two minor TODOs.
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    fc7308a View commit details
    Browse the repository at this point in the history
  2. WIP: in roledb, remove intermediate data format; #660

    - Rename and alter some schemas that really address delegations,
    to make that clear.
    - Do away with the ROLEDB_SCHEMA, an intermediate metadata format
    that is not necessary and which incorrectly flattens the delegation
    graph, and similar schemas.
    - Rewrite getters/setters in roledb to respect the delegation
    graph rather than assuming that delegated targets roles have only
    one delegation pointing to them (see Issue #660).
    - Add a variety of TODOs for later.
    - Clarify docstrings as a result of the above.
    
    reinterpreting metadata
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    fd86d04 View commit details
    Browse the repository at this point in the history
  3. WIP, DO NOT MERGE -- quick sketch of new schemas in formats.py

    This is mid-development, but I'm pushing it so that Aditya can see
    where things are and the general shape of things.
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    6464f7a View commit details
    Browse the repository at this point in the history
  4. DOC: improve formats.py module explanation/docstring

    Warn folks about the larger structures being near the end,
    make description a bit more readable, highlight matches() and
    check_match() funcs, emphasize that this module defines the
    data structures / formats used.
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    af271dc View commit details
    Browse the repository at this point in the history
  5. Revise formats.py: version numbers, paths, minor misses

    - remove re-definition of rolename_schema
    - use securesystemslib.formats.PATH_SCHEMA for all paths, rather
      than using RELPATH_SCHEMA, which implies a distinction that
      we do not actually make, and checks we do not actually perform.
    - use INTEGER_NATURAL_SCHEMA for lengths and metadata versions
    
    Excludes fileinfo-related adjustments of the above, as those will
    follow in a separate fileinfo-specific commit.
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    2705b1e View commit details
    Browse the repository at this point in the history
  6. PR revision: SIGNING_SCHEMA -> SIGNERS_SCHEMA

    The misleadingly-named ROLE_SCHEMA was renamed to SIGNING_SCHEMA,
    but I'm now making it SIGNERS_SCHEMA, which I think is clearer.
    
    I also added an example.
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    0aad781 View commit details
    Browse the repository at this point in the history
  7. PR revision: minor bugfix in ROLES_SCHEMA definition

    Failed to include key and value definitions.
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    42b14ef View commit details
    Browse the repository at this point in the history
  8. Rewrite FILEINFO schemas for clarity, and add comments

    In TUF, we store information about files in a variety of ways.
    Sometimes, versions are used, and sometimes length and hashes are required.
    So FILEINFO_SCHEMA will match any of these three new schemas:
    FILEINFO_IN_TIMESTAMP_SCHEMA, FILEINFO_IN_SNAPSHOT_SCHEMA,
    and FILEINFO_IN_TARGETS_SCHEMA.
    
    This should be more intuitive than the former mess, I think.
    
    I also renamed TARGETINFO to LABELED_FILEINFO_SCHEMA, with an
    explanation.  I hope that proves more intuitive as well.
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    4998117 View commit details
    Browse the repository at this point in the history
  9. DO NOT MERGE -- adds a TODO to roledb

    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 3, 2019
    Configuration menu
    Copy the full SHA
    a64fe91 View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2019

  1. PR revision: bugfix and rename roledb._is_top_level_role

    It'll now be a public function used by other modules (tuf.sig),
    so make it public and improve the name (takes a rolename, not a
    role):  roledb.is_top_level_rolename().
    
    Also bugfix it to handle casing issues.
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 15, 2019
    Configuration menu
    Copy the full SHA
    0c7c64e View commit details
    Browse the repository at this point in the history
  2. DOC: Simplify module docstring for tuf.sig

    It previously included information that wasn't really appropriate
    at this level of the code (about the project as a whole).
    
    Add short summary and list the two public functions with short
    explanations.
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 15, 2019
    Configuration menu
    Copy the full SHA
    9a578f8 View commit details
    Browse the repository at this point in the history
  3. Refactor and correct tuf.sig for Issue #660:

    - get_signature_status() and verify() will expect EITHER:
        - keyids AND threshold
        - a rolename
      never both.  A rolename can be used in place of keyids
      and threshold only for top-level roles, and keyids and
      threshold will then be drawn from currently trusted
      Root metadata.  See comments in the code for more.
      This includes making rolename an optional argument.
      Now uses tuf.roledb.is_top_level_rolename.
    - renamed "role" argument to "rolename", which is more
      correct (since the actual role metadata is another
      argument...)
    - Pulled the elaborate argument validation and the
      retrieval of keyids and threshold from Root metadata
      into a separate function:
        _determine_keyids_and_threshold_to_use
    - perform retrieval of keyids and threshold only from
      Root metadata, for top-level roles (part of #660)
    - removed unused generate_rsa_signature
    - cleaned up the structure of get_signature_status() a bit
    
    Tests will break and require fixes.
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 15, 2019
    Configuration menu
    Copy the full SHA
    1fcc652 View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2019

  1. PR revision: remove leftover code in get_signature_status:

    The keyids and threshold retrieval are already performed above now,
    so this lingering threshold retrieval is no longer needed.  Move
    the comment about errors it would raise to where that actually
    would happen now (and refine comment given new functionality).
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 16, 2019
    Configuration menu
    Copy the full SHA
    cd3d470 View commit details
    Browse the repository at this point in the history
  2. Remove tests for deleted functions in tuf.sig

    tuf.sig.generate_rsa_signature and tuf.sig.may_need_new_keys
    were not necessary and were deleted.  This commit removes their
    tests.
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 16, 2019
    Configuration menu
    Copy the full SHA
    43750ba View commit details
    Browse the repository at this point in the history
  3. minor: DOC: clarify comment in test_sig

    Explain the test conditions.
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 16, 2019
    Configuration menu
    Copy the full SHA
    cdae6b3 View commit details
    Browse the repository at this point in the history
  4. DO NOT MERGE: add TODOs to test_sig for after roledb changes

    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 16, 2019
    Configuration menu
    Copy the full SHA
    e6660cb View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2019

  1. Configuration menu
    Copy the full SHA
    fe8530f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7e5e90a View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3e3a94b View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    46b6d6e View commit details
    Browse the repository at this point in the history
  5. Fix variable name - typo

    adityasaky committed Apr 19, 2019
    Configuration menu
    Copy the full SHA
    ed2a486 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    d11ec7a View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    b710a18 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    c3f473f View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    b93122c View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    914c93b View commit details
    Browse the repository at this point in the history
  11. Fix lower() call

    adityasaky committed Apr 19, 2019
    Configuration menu
    Copy the full SHA
    0212f32 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2019

  1. Configuration menu
    Copy the full SHA
    339f7c4 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    773e7f6 View commit details
    Browse the repository at this point in the history
  3. Remove pdb statements

    adityasaky committed Apr 23, 2019
    Configuration menu
    Copy the full SHA
    8a86da9 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    0809280 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    e9b23eb View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    9398209 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    9569246 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    bb3bc15 View commit details
    Browse the repository at this point in the history
  9. Fix tuf.roledb reference

    adityasaky committed Apr 23, 2019
    Configuration menu
    Copy the full SHA
    d893b65 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    e9bd455 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    844a2ce View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    4e1f89a View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    b2d36c5 View commit details
    Browse the repository at this point in the history
  14. Add TODO for assertion fix

    adityasaky committed Apr 23, 2019
    Configuration menu
    Copy the full SHA
    24daeaf View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2019

  1. Configuration menu
    Copy the full SHA
    19ef4b9 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    ab6df47 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2019

  1. Remove stray pdb statement

    adityasaky committed Apr 26, 2019
    Configuration menu
    Copy the full SHA
    cf8f93e View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2019

  1. Configuration menu
    Copy the full SHA
    52d4923 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f9c351a View commit details
    Browse the repository at this point in the history
  3. PR Revision: Fix misleading error msg, roledb.get_delegation

    An error that is raised if someone tries to query a delegation
    that shouldn't exist (root to a delegated targets role or a
    delegated targets role to root, say) previously only described
    one direction, leading to misleading error messages.  It now
    explains both possible causes of the error.
    
    Also removes a pdb.set_trace() left over from prior revisions.
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 30, 2019
    Configuration menu
    Copy the full SHA
    ea3c4a9 View commit details
    Browse the repository at this point in the history
  4. Fix remaining test_roledb tests, reorganize arg tests

    Two functions now exist to replace _test_rolename (which was a bit
    of a misleading name), and these are now used to perform argument
    testing for roledb functions that query a single role or query
    information about a delegation from one role to another role.
    
    In addition, tests for roledb.get_delegated_paths were also
    updated, duplicating some of the above for reasons explained in
    code comments.
    
    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed Apr 30, 2019
    Configuration menu
    Copy the full SHA
    fdcf829 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    131afb8 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    0c7f8a1 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    3c5d182 View commit details
    Browse the repository at this point in the history

Commits on May 1, 2019

  1. Merge branch '660_fix_roledb_tests' into 660_respect_delegation_graph

    Signed-off-by: Sebastien Awwad <sebastien.awwad@gmail.com>
    awwad committed May 1, 2019
    Configuration menu
    Copy the full SHA
    fc872d7 View commit details
    Browse the repository at this point in the history