From c5efe748148f11a693ec0fcfed619a7edee92b24 Mon Sep 17 00:00:00 2001 From: David Souther Date: Wed, 14 Feb 2024 10:11:04 -0500 Subject: [PATCH] Delete unused and outdated extract-snippets --- .github/extract-snippets/README-SNIPPETS.txt | 3 - .github/extract-snippets/README.md | 268 ----------- .github/extract-snippets/extract-snippets.py | 453 ------------------ .github/extract-snippets/extract-snippets.sh | 34 -- .../snippet-extensions-more.yml | 44 -- .../extract-snippets/snippet-extensions.yml | 10 - 6 files changed, 812 deletions(-) delete mode 100644 .github/extract-snippets/README-SNIPPETS.txt delete mode 100644 .github/extract-snippets/README.md delete mode 100644 .github/extract-snippets/extract-snippets.py delete mode 100755 .github/extract-snippets/extract-snippets.sh delete mode 100644 .github/extract-snippets/snippet-extensions-more.yml delete mode 100644 .github/extract-snippets/snippet-extensions.yml diff --git a/.github/extract-snippets/README-SNIPPETS.txt b/.github/extract-snippets/README-SNIPPETS.txt deleted file mode 100644 index b259f027706..00000000000 --- a/.github/extract-snippets/README-SNIPPETS.txt +++ /dev/null @@ -1,3 +0,0 @@ -The snippet files in this directory are extracted from source files elsewhere -in this repository by an automated process. Do not make changes to snippets -yourself; your changes will likely be lost in the not-so-distant future. \ No newline at end of file diff --git a/.github/extract-snippets/README.md b/.github/extract-snippets/README.md deleted file mode 100644 index 3800ea7ceff..00000000000 --- a/.github/extract-snippets/README.md +++ /dev/null @@ -1,268 +0,0 @@ -# Extract Snippets for GitHub Actions - -Jerry Kindall, Amazon Web Services -Last updated 24-Jun-2021 - -## What it is - -This is a suite of GitHub Actions workflows with supporting scripts that -extracts code snippets from source code files, which can then be used in -documentation via include directives. When the code changes, the snippets are -automatically extracted again, and the documentation will pick them up on the -next build. - -There are two separate workflows: - -- Extract Snippets (`extract-snippets.yaml`): Extracts snippets from all source - files in the repo. Runs on a commit to the main or master branch; can also be - run manually. - -- Extract Snippets Dry Run (`extract-snippets-dryrun.yml`): Extracts snippets - from all source files in a pull request but does not check in any snippets; - meant to validate PRs. Displays only the issue report (problems found in the - extraction process). Can also be run manually. - -To prevent the introduction of consistency problems in the snippets (e.g. -duplicate snippet filenames with different content), all files in the repo are -always processed. This is not noticeably slower than e.g. processing only the -files in a given commit; the overhead of the action setup and Git commands -overshadows the run time of the actual snippet extraction. - -## Compared to other snippet extraction tools - -The AWS Docs organization has a tool which it uses to extract snippets from -source files. Compared to that tool, this tool has the following additional -features: - -- Runs on GitHub so snippets can be automatically updated on every commit. -- Includes snippet-append and snippet-echo tags (see below). -- Can dedent (remove indentation from) extracted snippets. -- Checks for and logs a variety of problems, including conflicting snippet - tags (same tag in multiple files). -- Besides a processing log, produces a report of files with problems and an - index mapping snippet tags back to the file(s) that contain them. - -It does not have the following features of the AWS Docs tool: - -- Extract metadata from snippets for use in catalogs. The metadata tags are - recognized, but do not do anything, in this snippet extractor. - -## Snippet tags - -Snippet tags are special single-line comments in source files. They must not -follow any code on their line and must begin with the language's single-line -comment marker (`//` in many languages, `#` in some others). If a language does -not have a single-line comment marker, the block comment delimiter may be used, -but should be closed on the same line following the snippet tag. The snippet -tag is followed by the snippet directive, a colon, and an argument in square -brackets. Whitespace is permitted (but optional) between the comment marker and -the snippet directive. For example: - -Here, the directive begins the extraction of a code snippet to the filename -specified, with a `.txt` extension. - -The main tags used in our repos are `snippet-start` and `snippet-end`. Each -`snippet-start` requires a matching `snippet-end` (specifying the same snippet -name) in the same source file. Multiple snippets may be extracted from one -source file, and may overlap. The snippet tags do not appear in the extracted -snippets. - -The following tags are unique to this extractor (they are not supported by the -snippet extractor used by the AWS Docs team). - -- `snippet-append`: Extracts additional source code to a snippet file that has - already been created by a previous `snippet-start` directive, stopping at - `snippet-end` as with `snippet-start`. - -- `snippet-echo`: Writes the argument literally to the snippet(s) currently - being extracted. Useful for adding closing braces etc. when extracting a - partial code block. Whitespace is stripped from the right of the argument but - not the left, so you can match indentation. - -Also unique to this extractor, `snippet-start` supports an optional number -following the closing bracket. - -If this number is present, that many spaces are removed from the beginning of -each line of the snippet, allowing snippets to be dedented (have indentation -removed), so their left margin is decreased. Each snippet, even overlapping -snippets, has its own dedent level. If you use `snippet-append`, it uses the -same dedent specified on `snippet-start`. Dedent does not affect -`snippet-echo`, so provide the desired indentation yourself. - -This extractor also recognizes the following tags (i.e. they are not errors), -but does not do anything with them. They are supported for compatibility with -source files tagged for the original AWS Docs extractor, which can register -metadata about each snippet. - -- `snippet-keyword` -- `snippet-service` -- `snippet-sourceauthor` -- `snippet-sourcedate` -- `snippet-sourcedescription` -- `snippet-sourcesyntax` -- `snippet-sourcetype` - -## extract-snippets.sh - -This `bash` script calls the Python script (described next) to extract the -snippets, then checks the results in to the `snippets` branch of the repo. If -the script is passed any argument (value is irrelevant), it exits after -extracting the snippets without adding them to the repo ("dry run" mode). - -## extract-snippets.py - -This script reads from standard input the paths of the files containing the -snippets to be extracted. It ignores non-source files, hidden files, and files -in hidden directories (it is not necessary to filter out such files beforehand). -The script's required argument is the directory that the snippets should be -extracted into. This directory must not contain any files named the same as a -snippet being extracted. - -For example, the following command runs the script on source files in the -current directory, extracting snippets also into the current directory. - -``` -ls | python3 extract-snippets.py . -``` - -Both Windows and Linux-style paths are supported so you can test the script on -Windows during development. - -The supported source file formats are stored in `snippet-extensions.yml` or -another file specified as the second command-line argument. This file is a YAML -map of filename extensions to comment markers. If a language supports more than -one line comment marker, you can provide them separated by whitespace in a -single string: - -``` -.php: "# //" -``` - -If a language does not support a line comment marker (e.g. C), you can specify -its starting block comment marker. However, the extraction process does not -include the lines with the snippet tags in the snippets, so you should include -the closing block comment marker on the same line to avoid the closing marker -being included in the snippet. For example: - -/_ snippet-start:[terry.riley.in-c] _/ - -Not: - -/_ snippet-start:[terry.riley.in-c]
-_/ - -Some languages support both line and block comments. In this case, we suggest -you always use the line comment marker for snippet tags. - -You may pass a different YAML (or JSON) file as the script's second argument -- -for example, the provided `snippet-extensions-more.yml`, which contains a more -extensive map of source formats. Note that if you specify only a filename, the -file of that name _in the same directory as the script_ (not in the working -directory!) is used. To specify a file in the current directory, use `./`, e.g. -`./my-snippet-extensions.yml`. - -The keys in `snippet-extensions.yml` are matched case-sensitively at the end of -file paths, and can be used to match more than extensions. If you wanted to -extract snippets from makefiles, for example, you could add to the mapping: - -``` -/makefile: "#" -``` - -The slash makes it match the complete filename: i.e., there is a directory -separator, then "makefile", at the end of the path. Always use `/` for this -purpose even if you are using Windows paths with backslashes; paths are -normalized to use slashes before this comparison. - -If a given path could match more than one language, the first one listed in the -extension file wins. - -To match all files, use `""` as the key (after all, there's an empty string at -the end of every path -- in fact, infinitely many of them). You probably -shouldn't do this, but you _can._ It might be useful as a catch-all in a repo -where you want to process all files and most languages in the repo use the same -comment marker. It should go last in the extensions file. - -To exclude a file or files from being processed, specify the end of its path and -an empty string as the comment marker. Such items should appear earlier in the -file than others that might match, since the first match wins. - -``` -"/lambda/widgets.js": "" -``` - -The output of `extract-snippets.py` is a list of the source files processed. -Indented under each source file is a list of the snippets extracted from it, if -any, notated with EXTRACT. APPEND operations, errors, and warnings are also -flagged in similar fashion under the source file. - -At the end of the run, a summary line displays the number of unique snippets -extracted and the number of source files examined. This is followed by a report -of all files with issues, and an index that maps snippets back to the files that -contain them. - -## Errors - -The following situations are considered problematic to varying degrees. To the -extent possible, errors do not stop processing. - -- Unrecognized snippet tag (see earlier section for supported tags). - -- Text decoding error. By default, source files are assumed to be UTF-8. To - change the encoding used, sent the environment variable `SOURCE_ENCODING` to - `utf16` or another encoding. Use the Python name, which you can find here: - - https://docs.python.org/3/library/codecs.html#standard-encodings - - Generally you'd do this in the action file, not in the `bash` script. Like: - -```yaml -# goes under the `steps` key -env: - SOURCE_ENCODING: utf-16 -``` - -If a file cannot be read, processing continues with the next file, if any. - -- `snippet-start` for a snippet file that has already been extracted, _unless_ - the source file has the same filename and contains exactly the same code. This - behavior supports multiple examples that contain the same source code for an - incorporated Lambda function or other asset, where that code contains - snippets. The former situation is an error, the latter a warning. - -- `snippet-end` with no corresponding `snippet-start` or `snippet-append` in the - same source file. - -- Missing `snippet-end` corresponding to a `snippet-start` or `snippet-append`. - -- `snippet-append` with no corresponding `snippet-start` in the same source file - (you can't append to snippets created in a different source file since there's - no guarantee the files will be processed in any particular order, or that all - files will even be processed, leading to consistency issues). - -- `snippet-echo` outside of a snippet. - -- Insufficient whitespace at the beginning of a line to dedent it as required. - -- Any source file contains both a tag and a tab character (ASCII 9), as - indenting by tab is not well-supported in documentation. This is a warning, - and will not on its own stop extracted snippets from being checked in. - -If there is at least one error, none of the extracted snippets will be checked -in. Warnings _do not_ prevent snippets from being checked in. - -# README-SNIPPETS.txt - -This text file is copied into the snippets directory as README.txt and should -provide information that users of the snippets should know. - -# Version history - -- v1.0.0 - Initial pull request against the CDK Examples repo. -- v1.1.0 - Test against SDK team's examples repo and fix the problems found. - Continue processing after errors instead of stopping at the first. - Generate report of files with issues. - Generate index of files containing each snippet. - Other fixes and tweaks. -- v1.2.0 - Allow log, issue report, and index to be selectively enabled. - Don't try to call non-method attributes to handle snippet tags. diff --git a/.github/extract-snippets/extract-snippets.py b/.github/extract-snippets/extract-snippets.py deleted file mode 100644 index 688810adc1b..00000000000 --- a/.github/extract-snippets/extract-snippets.py +++ /dev/null @@ -1,453 +0,0 @@ -# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. -# SPDX-License-Identifier: Apache-2.0 -# extract-snippets.py v1.2.0 6/24/2021 -# Jerry Kindall, Amazon Web Services - -# extracts tagged regions from source files and writes them to a snippets directory. -# reads list of paths from stdin and extracts snippets from these files. Takes the -# directory to which snippets should be extracted as a command line argument. The -# second command line argument is optional and specifies the YAML file that contains -# a map of filename extensions to comment markers (default: snippet-extensions.yml -# in the same directory as this script) - -# examples: -# -# extract snippets from last commit on current git branch to /tmp -# git diff @^ --name-only | python3 extract-snippets.py /tmp -# -# extract snippets from all files to specified directory -# find . -type f | python3 extract-snippets.py /path/to/snippets/dir -# -# extract snippets from all files in current dir to current dir, -# specifying a different filename map -# ls | python3 extract-snippets.py . snippet-extensions-more.yml - -# The same snippet can be extracted from more than one source file ONLY if all -# source files containing the snippet have the same filename and contents. -# this is to support e.g. Lambda functions deployed by a CDK example, where the -# CDK app is provided in multiple languages but the same Lambda function source -# code (snippet tags included) is used in each version. Ideally the snippet tags -# would be removed from all but one of the Lambda source files... ideally. - -# This script also presents an index mapping snippet names back to the files -# they come from, and a list of source files having problems. - -import sys, os, io, yaml, re, functools - -# all open() calls have an implied encoding parameter, UTF-8 by default -open = functools.partial( - __builtins__.open, encoding=os.environ.get("SOURCE_ENCODING", "utf8") -) - -# some constants to make our lives easier -TAB = "\t" -EOL = "\n" - -# regular expression for matching dedent specifier: 1 or 2 digits -DIGITS = re.compile("[0-9][0-9]?") - - -# returns cached contents of a file if it exists, or reads it into the cache and -# returns it if not. cache is stored as a default parameter value. -# -# the cache is used only when there are duplicate snippets in two or more source files. -# only one copy of the file is ever cached (the first one that was found) so this shouldn't -# run up memory too much if you don't have many duplicate snippets. -def cached(path, cache={}): - if path not in cache: - with open(path) as infile: - cache[path] = infile.read().rstrip() - return cache[path] - - -# a file-like object used to avoid writing duplicate snippets we've already extracted -# in situations where this is not an error -class DummyFile: - def __init__(self, *args, **kwargs): - pass - - def write(self, text): - pass - - def close(self): - pass - - -# auto-vivifying dict (like DefaultDict but we don't need to import it) -class AutoDict(dict): - def __init__(self, T): - self.T = T - - def __missing__(self, key): - self[key] = self.T() - return self[key] - - -# the class that does the snippet extraction. instantiate it passing the directory to -# which snippets should be extracted. call the instance with each source file. -class Snipper: - # initialize Snipper - def __init__(self, snippetdir): - self.dir = snippetdir # directory where snippets will be extracted - self.source = {} # source file of each snippet - self.count = 0 # number of snippets extracted - self.errors = 0 # processing errors - self.issues = AutoDict(set) # files with issues - self.index = AutoDict( - list - ) # index of snippets to files (this should probably be merged with self.source) - self.log = io.StringIO() - - # if used as context manager, we capture the log instead of printing it as we go - # by switching print() to print to a StringIO object - def __enter__(self): - global print - print = functools.partial(__builtins__.print, file=self.log) - return self - - def __exit__(self, *args): - global print - print = __builtins__.print - - # extract snippets from a single file - def __call__(self, path, markers): - print(path) - self.started = set() # snippets we've started in this source file - self.duplicates = ( - set() - ) # snippets we've determined are duplicates so we won't append/echo - tag = re.compile( - f" *({'|'.join(markers)}) ?snippet-" - ) # e.g. if ext is "// #" end up with regex: " *(#|//) ?snippet-" - self.files = {} # files currently open to write snippets - self.dedent = {} # amount of whitespace to strip from each line of snippet - self.path = path # source file we are working with (store it on instance so we can use it in error messages) - self.markers = markers - try: - with open(path) as infile: # read source file entirely into memory - self.text = infile.read().rstrip() - except IOError as ex: - print("ERROR reading file", ex) - self.errors += 1 - return - if TAB in self.text and "snippet-start" in self.text: - print( - " WARNING tab(s) found in %s may cause formatting problems in docs" - % path - ) - # process each line in source file. self.i is the line we're on (for error messages) - for self.i, self.line in enumerate( - self.text.splitlines(keepends=False), start=1 - ): - line = self.line # use a local variable for a bit more performance - if tag.match(line): # line is a snippet directive, parse and process it - self.directive = ( - line.split("snippet-")[1].split(":")[0].rstrip() - ) # get e.g. append fron snippet-append - self.arg = ( - line.split("[")[1].split("]")[0].rstrip() - ) # get e.g. snippet-name from [snippet-name] - func = getattr(self, self.directive.lstrip("_"), None) - if func and callable(func): - func( - self.arg - ) # call our method named same as directive (e.g. start(..) for snippet-start) - else: - print( - " ERROR invalid directive snippet-%s at %s in %s" - % (self.directive, self.i, self.path) - ) - self.errors += 1 - self.issues[path].add( - "invalid directive snippet-%s" % self.directive - ) - else: # line is NOT a snippet directive. write it to any open snippet files - for ( - snip, - file, - ) in ( - self.files.items() - ): # for each snippet file we're writing, write the line - dedent = self.dedent[snip] - if ( - dedent and line[:dedent].strip() - ): # is the text we want to strip to dedent all whitespace? error if not - print( - (" ERROR unable to dedent %s space(s) " % dedent) - + ("in snippet %s at line %s in %s " % self._where) - + f"(only indented {len(line) - len(line.lstrip())} spaces)" - ) - self.errors += 1 - file.write( - line[dedent:].rstrip() + EOL - ) # write it (strip whitespace at end just to be neat) - # done processing this file. make sure all snippets had snippet-end tags - for snip, file in self.files.items(): - print( - " ERROR snippet-end tag for %s missing in %s, extracted to end of file" - % (snip, path) - ) - file.close() - self.issues[path].add("snippet-end tag for %s missing" % snip) - self.errors += 1 - - # directive: beginning of snippet - def start(self, arg): - path = os.path.join(self.dir, f"{arg}.txt") - indicator = "EXTRACT" - opener = open - printer = print - if arg in self.files: - printer = lambda *a: print( - " ERROR snippet %s already open at line %s in %s" % self._where - ) - self.issues[self.path].add("snippet %s opened multiple times") - self.errors += 1 - elif os.path.isfile(path): - # if snippet output already exists, this is OK only if it source file has the same name and identical content - if ( - self.path != self.source[arg] - and self.path.rpartition("/")[2] == self.source[arg].rpartition("/")[2] - and self.text == cached(self.source[arg]) - ): - printer = lambda *a: print( - "WARNING redundant snippet %s at line %s in %s" % self._where - ) - self.duplicates.add(arg) - else: - printer = lambda *a: print( - " ERROR duplicate snippet %s at line %s in %s" % self._where, - "(also in %s)" % self.source[arg], - ) - pfxlen = len(os.path.commonprefix([self.path, self.source[arg]])) - path1 = self.source[arg][pfxlen:] - if "/" not in path1: - path1 = self.source[arg] - path2 = self.path[pfxlen:] - if "/" not in path2: - path2 = self.path - self.issues[self.path].add("%s also declared in %s" % (arg, path1)) - self.issues[self.source[arg]].add( - "%s also declared in %s" % (arg, path2) - ) - self.errors += 1 - opener = DummyFile # don't write to the file, but still track it so we can detect missing snippet-end - else: - self.count += 1 - # parse number at end of line as dedent value - self.dedent[arg] = int( - DIGITS.search(self.line.rpartition("]")[2] + " 0").group(0) - ) - self.files[arg] = opener(path, "w") # open real file or dummy - self.index[arg].append(self.path) - self.started.add(arg) # record that we started this snippet in this source file - if ( - arg not in self.source - ): # record that we *first* saw this snippet in this source file - self.source[arg] = self.path - printer(" ", indicator, arg) - - # directive: append to given file (for extracting multiple chunks of code to a single snippet) - def append(self, arg): - if arg in self.files: # is the file already open? - print(" ERROR snippet %s already open at line %s in %s" % self._where) - self.issues[self, path].add("snippet %s opened multiple times" % arg) - self.errors += 1 - return - if arg not in self.started: # did we start this snippet in current source file? - print(" ERROR snippet file %s not found at line %s in %s" % self._where) - self.issues[self.path].add("snippet %s doesn't exist" % arg) - self.errors += 1 - return - self.files[arg] = ( - DummyFile() - if arg in self.duplicates - else open(os.path.join(self.dir, arg) + ".txt", "a") - ) - print(" APPEND", arg) - - # directive: end of snippet - def end(self, arg): - if arg in self.files: - self.files[arg].close() - del self.files[arg] - else: - print(" ERROR snippet file %s not open at %s in %s" % self._where) - self.issues[self.path].add("snippet-end tag for %s which is not open" % arg) - self.errors += 1 - - # directive: insert arg verbatim as a line into all currently open snippets - # useful for e.g. adding closing brackets to partial code block (could also use append for that) - def echo(self, arg): - arg = arg.rstrip() + EOL - if self.files: - for file in self.files.values(): - file.write(arg) - else: - print(" ERROR echo '%s' outside snippet at %s in %s" % self._where) - self.issues[self.path].add("echo outside snippet") - self.errors += 1 - - # do-nothing handler used for directives that we ignore - def _nop(self, arg): - return - - # the aforementioned ignored directives - service = ( - comment - ) = ( - keyword - ) = sourceauthor = sourcedate = sourcedescription = sourcetype = sourcesyntax = _nop - - # convenience property for returning error location tuple (used in error messages) - @property - def _where(self): - return self.arg, self.i, self.path - - -def err_exit(msg): - print("ERROR", msg) - sys.exit(1) - - -# ---------------------------------------------------------------------------- - -if __name__ == "__main__": - # read list of filenames from stdin first, so we don't get broken pipe if we error out - stdin_lines = [] - if not sys.stdin.isatty(): - stdin_lines = sys.stdin.readlines() - - # get output directory from command line, or error - if len(sys.argv) > 1 and os.path.isdir(sys.argv[1]): - snippetdir = sys.argv[1] - else: - err_exit("snippet output directory not passed or does not exist") - - # get filename of extersions list from command line, or use default, then load it - if len(sys.argv) > 2: - commentfile = sys.argv[2] - else: - commentfile = "snippet-extensions.yml" - - # reports to be printed can be passed in via environment variable REPORTS - # if this value is not set, print all reports - reports = os.environ.get("REPORTS", "log issues index").lower().split() - - # if no directory specified, file is in same directory as script - if "/" not in commentfile and "\\" not in commentfile: - commentfile = os.path.join(os.path.dirname(__file__), commentfile) - if not os.path.isfile(commentfile): - err_exit("source file extension map %s not found" % commentfile) - with open(commentfile) as comments: - MAP_EXT_MARKER = yaml.safe_load(comments) - if not isinstance(MAP_EXT_MARKER, dict): - err_exit("source map is not a key-value store (dictionary)") - for k, v in MAP_EXT_MARKER.items(): - if isinstance(k, str) and isinstance(v, str): - MAP_EXT_MARKER[k] = v.split() - else: - err_exit( - "key, value must both be strings; got %s, %s (%s, %s)" - % (k, v, type(k).__name__, type(v).__name__) - ) - - print( - "==== extracting snippets in source files", - " ".join(ex for ex in MAP_EXT_MARKER if ex and MAP_EXT_MARKER[ex]), - "\n", - ) - print("reports:", " ".join(reports).upper(), end="\n\n") - - # initialize snipper instance and our counters - with Snipper(snippetdir) as snipper: - seen = processed = 0 - - # main loop: for each file named on stdin, check to see if we should process it, and if so, do so - for path in sorted(stdin_lines): - path = path.strip() - if not path: # skip blank lines in input - continue - # make sure relative path starts with ./ so that e.g. /Makefile in the extensions map - # can be used to match an entire filename. - if not ( - path.startswith(("./", "/", "\\")) - or ( # already relative or Linux/Mac absolute path or UNC path - path[0].isalpha() and path[1] == ":" - ) - ): # already Windows absolute path - path = "./" + path - if "/." in path or "\\." in path: # skip hidden file or directory - continue - seen += 1 # count files seen (not hidden) - # find first extension from extension map that matches current file - # replace backslashes with forward slashes for purposes of matching so it works with Windows or UNC paths - ext = next( - ( - ext - for ext in MAP_EXT_MARKER - if path.replace("\\", "/").endswith(ext) - ), - None, - ) - markers = MAP_EXT_MARKER.get(ext, ()) - if markers: # process it if we know its comment markers - snipper(path, markers) - processed += 1 - - # files with issues report (files with most issues first) - if "issues" in reports: - if snipper.issues: - print("====", len(snipper.issues), "file(s) with issues:", end="\n\n") - for issue, details in sorted( - snipper.issues.items(), key=lambda item: -len(item[1]) - ): - print(issue, end="\n ") - print(*sorted(details), sep="\n ", end="\n\n") - else: - print("---- no issues found\n") - - # snippet index report (snippets that appear in the most files first) - if "index" in reports: - if snipper.index: - print( - "====", - len(snipper.index), - "snippet(s) extracted from", - processed, - "files:", - end="\n\n", - ) - for snippet, files in sorted( - snipper.index.items(), key=lambda item: -len(item[1]) - ): - print(snippet, "declared in:", end="\n ") - print(*sorted(files), sep="\n ", end="\n\n") - else: - print("--- no snippets were extracted\n") - - # print log - if "log" in reports: - print("==== Complete processing log\n") - if processed: - print(snipper.log.getvalue(), end="\n\n") - else: - print("No files were processed\n") - - # print summary - print( - "====", - snipper.count, - "snippet(s) extracted from", - processed, - "source file(s) processed of", - seen, - "candidate(s) with", - snipper.errors, - "error(s) in", - len(snipper.issues), - "file(s)\n", - ) - - # exit with nonzero status if we found any errors, so caller won't commit the snippets - sys.exit(snipper.errors > 0) diff --git a/.github/extract-snippets/extract-snippets.sh b/.github/extract-snippets/extract-snippets.sh deleted file mode 100755 index 0b79798873a..00000000000 --- a/.github/extract-snippets/extract-snippets.sh +++ /dev/null @@ -1,34 +0,0 @@ -# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. -# SPDX-License-Identifier: Apache-2.0 -# extract-snippets.sh v1.1.0 -# Jerry Kindall, Amazon Web Services - -# this script manages updating of snippets found in source code to the snippets branch - -# the args to this script constitute the command that is executed to get the list of source -# files to be processed. for example, if it is run on push, pass e.g.: git diff @^ --name-only -# to process only the files included in the last push. to process all files, pass e.g.: -# find . --type f - -# snippets are never deleted, as this would break any document where they're used. - -mkdir ../snippets -cp .github/extract-snippets/README-SNIPPETS.txt ../snippets/README.txt - -python -m pip install --upgrade pip pyyaml -find . -type f | python .github/extract-snippets/extract-snippets.py ../snippets snippet-extensions-more.yml || exit 1 - -if [[ -z $1 ]]; then - - git checkout --track origin/snippets - mkdir -p snippets - cp ../snippets/* snippets - - git config --local user.email "41898282+github-actions[bot]@users.noreply.github.com" - git config --local user.name "github-actions[bot]" - git add --all - git commit -m "Updated on `date -R`" - -fi - -exit 0 \ No newline at end of file diff --git a/.github/extract-snippets/snippet-extensions-more.yml b/.github/extract-snippets/snippet-extensions-more.yml deleted file mode 100644 index 995eb9f4df9..00000000000 --- a/.github/extract-snippets/snippet-extensions-more.yml +++ /dev/null @@ -1,44 +0,0 @@ -# This file maps filename extensions (or, really, any string ending a path, -# such as /makefile) to comemnt markers for a particular language, so that -# snippet directives can be detected in that source file - -somewhere/lambda/widgets.js: "" # example of excluding a file via empty comment marker - -.ahk: ; # AutoHotkey -.bash: "#" # bash shell (must quote "#" here as it is also the YAML comment marker) -.bat: "REM @REM rem @rem Rem @Rem" # Windows batch file -.class: // # Java -.c: "/*" # C (closing comment marker must be on same line) -.cpp: // # C++ -.csh: "#" # C shell -.cs: // # C# -.dart: // # Dart -.f: "* c !" # FORTRAN (fixed field; most compilers also support ! even in fexed field) -.f90: "!" # FORTRAN (freeform) -.fs: // # F# -.h: "/* //" # C or Objective-C header (closing comment marker must be on same line) -.hpp: // # C++ header -.go: // # Go -.html: