Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Markdown documentation generator #963

Merged
merged 48 commits into from
Apr 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
0bc0ca5
refactor: rename a parameter
lars-reimann Mar 31, 2024
761c28a
refactor: move Python generation tests into new subfolder
lars-reimann Mar 31, 2024
f15a764
feat: add empty `MarkdownGenerator` service
lars-reimann Mar 31, 2024
784d343
test: data-driven tests for markdown generator
lars-reimann Mar 31, 2024
aa2cc07
feat: get description and value of `@since` tag of declaration
lars-reimann Mar 31, 2024
aa72777
feat: add `partitionBy` utility function
lars-reimann Mar 31, 2024
19ae231
build(deps-docs): add `mkdocs-literate-nav`
lars-reimann Mar 31, 2024
ee69229
feat: create summary page for navigation
lars-reimann Mar 31, 2024
6ae3c4f
build(deps-docs): add `mkdocs-autorefs`
lars-reimann Mar 31, 2024
8256e86
feat: generate documentation for schemas
lars-reimann Mar 31, 2024
674de33
feat: render source code
lars-reimann Mar 31, 2024
d2464ec
feat: generate documentation for annotations
lars-reimann Apr 1, 2024
30674d5
feat: generate documentation for enums
lars-reimann Apr 1, 2024
63c8130
feat: generate documentation for segments
lars-reimann Apr 1, 2024
8bdb180
feat: generate documentation for functions
lars-reimann Apr 1, 2024
48adefc
feat: syntax highlighting for default values of parameters
lars-reimann Apr 1, 2024
2f4b003
feat: generate documentation for classes
lars-reimann Mar 31, 2024
1bf9b6c
feat: add `document` command to CLI
lars-reimann Apr 1, 2024
8940266
fix: normalize newlines
lars-reimann Apr 1, 2024
731449e
test: drastically speed up scoping tests
lars-reimann Apr 1, 2024
de05df6
docs: fix wrong method names
lars-reimann Apr 1, 2024
d777cf7
fix: building of summary page
lars-reimann Apr 1, 2024
0055102
docs: fix Mathjax startup
lars-reimann Apr 1, 2024
ec9d43a
fix: icon for experimental declarations
lars-reimann Apr 1, 2024
1e20e40
docs: fix typo
lars-reimann Apr 1, 2024
db68574
test: delete file that are generated by CLI
lars-reimann Apr 1, 2024
a85400d
fix: indentation of source code
lars-reimann Apr 1, 2024
4e982b8
fix: keyword of functions
lars-reimann Apr 1, 2024
d875e98
fix: word break of code elements in table cells
lars-reimann Apr 1, 2024
00d00bb
refactor: move methods to add/remove line prefix to helpers
lars-reimann Apr 1, 2024
ab654cb
fix: linking when creating docs for stdlib
lars-reimann Apr 1, 2024
c4366a7
fix: remove syntax highlighting for named types
lars-reimann Apr 1, 2024
1b171e3
feat: show targets of annotations
lars-reimann Apr 1, 2024
b88b66d
fix: rendering of Math blocks
lars-reimann Apr 1, 2024
8d4c5ea
fix: package of some stub files
lars-reimann Apr 1, 2024
c63ebda
test: add new snapshots
lars-reimann Apr 1, 2024
cd8741e
fix: text of tags can now span multiple lines
lars-reimann Apr 1, 2024
9eb6244
fix: rendering of inline link tags
lars-reimann Apr 1, 2024
0ef5c9b
test: improve coverage
lars-reimann Apr 1, 2024
f613d12
docs: generate docs for standard library
lars-reimann Apr 1, 2024
9cf3272
style: apply automated linter fixes
megalinter-bot Apr 1, 2024
4f62b76
docs: remove old files
lars-reimann Apr 1, 2024
cf07b5c
test: add dummy segment to fix test failure in CI
lars-reimann Apr 1, 2024
daafd81
docs: add missing `@param` tags
lars-reimann Apr 1, 2024
62a039e
feat: sort enum members by name
lars-reimann Apr 1, 2024
af85458
feat: sort class members
lars-reimann Apr 1, 2024
31aa174
docs: regenerate stdlib docs
lars-reimann Apr 1, 2024
a4e0320
fix: make `InvertibleTableTransformer` abstract
lars-reimann Apr 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
42 changes: 35 additions & 7 deletions docs/development/generation-testing.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
# Generation Testing

Generation tests are data-driven instead of being specified explicitly. This document explains how to add a new
generation test.
Safe-DS has two generation targets, [Python](#python-generation) (for execution) and [Markdown](#markdown-generation)
(for documentation). In either case, the generation tests are data-driven instead of being specified explicitly. This
document explains how to add a new generation test.

## Adding a generation test
## Python Generation

1. Create a new **folder** (not just a file!) in the `tests/resources/generation` directory or any subdirectory. Give
the folder a descriptive name, since the folder name becomes part of the test name. By default, the runner
### Adding a generation test

1. Create a new **folder** (not just a file!) in the `tests/resources/generation/python` directory or any subdirectory.
Give the folder a descriptive name, since the folder name becomes part of the test name. By default, the runner
integration is disabled. If you want to test the runner integration, place the folder in
`tests/resources/generation/runner integration` or any subdirectory instead.
`tests/resources/generation/python/runner integration` or any subdirectory instead.

!!! tip "Skipping a test"

Expand All @@ -30,6 +33,31 @@ generation test.
and the contents of the Python files will be compared to the actual generation output.
6. Run the tests. The test runner will automatically pick up the new test.

## Updating the expected output
### Updating the expected output

To quickly update the expected output after changes to the code generator, run `vitest` with the `--update` flag.

## Markdown Generation

### Adding a generation test

1. Create a new **folder** (not just a file!) in the `tests/resources/generation/markdown` directory or any
subdirectory. Give the folder a descriptive name, since the folder name becomes part of the test name.

!!! tip "Skipping a test"

If you want to skip a test, add the prefix `skip-` to the folder name.

2. Add files with the extension `.sdstest`, `.sdspipe`, or `.sdsstub` **directly inside the folder**. All files in a
folder will be loaded into the same workspace, so they can reference each other. Files in different folders are
loaded into different workspaces, so they cannot reference each other. Generation will be triggered for all files in
the folder.
3. Add the Safe-DS code that you want to test to the files.
4. Add another folder called `generated` inside the folder that you created in step 1. Place folders and Python files
inside the `generated` folder to specify the expected output of the program. The relative paths to the Python files
and the contents of the Python files will be compared to the actual generation output.
5. Run the tests. The test runner will automatically pick up the new test.

### Updating the expected output

To quickly update the expected output after changes to the code generator, run `vitest` with the `--update` flag.
3 changes: 3 additions & 0 deletions docs/javascript/mathjax.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,8 @@ window.MathJax = {
};

document$.subscribe(() => {
MathJax.startup.output.clearCache();
MathJax.typesetClear();
MathJax.texReset();
MathJax.typesetPromise();
});
11 changes: 4 additions & 7 deletions docs/language/stub-language/annotations.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ To attach metainformation to a declaration, the annotation must be called on tha
- [Type parameters][type-parameters]
- [Pipelines][pipelines]

The valid targets of an annotation can be restricted with the [`Target`][safeds-lang-target] annotation. By default all targets are allowed. Likewise, an annotation can only be called once on the same declaration by default, unless the annotation is marked as[Repeatable][safeds-lang-repeatable].
The valid targets of an annotation can be restricted with the [`Target`][safeds.lang.Target] annotation. By default all targets are allowed. Likewise, an annotation can only be called once on the same declaration by default, unless the annotation is marked as[Repeatable][safeds.lang.Repeatable].

Annotation calls are always located right in front of their target. Exception: In the case of compilations units they are located at the very top of the file. Here is an example that demonstrates how to call the annotation `#!sds OnlyForExperts` that we defined above on a [class][classes]:

Expand Down Expand Up @@ -87,10 +87,10 @@ The same [restrictions to arguments][argument-restrictions] as for [calls][calls

## Built-in Annotations

The package `#!sds safeds.lang` contains several annotations that are processed by Safe-DS. Refer to the [API documentation][safeds-lang] for more details. Particularly important are the annotations
The package `#!sds safeds.lang` contains several annotations that are processed by Safe-DS. Particularly important are the annotations

- [`Target`][safeds-lang-target], which can restrict the possible targets of an annotation, and
- [`Repeatable`][safeds-lang-repeatable], which allows an annotation to be called multiple times on the same declaration.
- [`Target`][safeds.lang.Target], which can restrict the possible targets of an annotation, and
- [`Repeatable`][safeds.lang.Repeatable], which allows an annotation to be called multiple times on the same declaration.

[parameters]: ../common/parameters.md
[types]: ../common/types.md
Expand All @@ -104,8 +104,5 @@ The package `#!sds safeds.lang` contains several annotations that are processed
[segments]: ../pipeline-language/segments.md
[type-parameters]: type-parameters.md
[pipelines]: ../pipeline-language/pipelines.md
[safeds-lang]: ../../stdlib/safeds_lang.md
[safeds-lang-repeatable]: ../../stdlib/safeds_lang.md#annotation-repeatable
[safeds-lang-target]: ../../stdlib/safeds_lang.md#annotation-target
[calls]: ../pipeline-language/expressions.md#calls
[argument-restrictions]: ../pipeline-language/expressions.md#restrictions-for-arguments
10 changes: 7 additions & 3 deletions docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,7 @@ nav:
- Enumerations: language/stub-language/enumerations.md
- Annotations: language/stub-language/annotations.md
- Type Parameters: language/stub-language/type-parameters.md
- Standard Library:
- stdlib/README.md
- safeds.lang: stdlib/safeds_lang.md
- Standard Library: stdlib/
- Development:
- Call Graph Testing: development/call-graph-testing.md
- Formatting Testing: development/formatting-testing.md
Expand Down Expand Up @@ -80,6 +78,9 @@ theme:
- navigation.top

plugins:
- literate-nav:
nav_file: SUMMARY.md
- autorefs
- glightbox
- same-dir
- search
Expand Down Expand Up @@ -130,6 +131,9 @@ markdown_extensions:
- toc:
permalink: true

extra_css:
- stylesheets/extra.css

extra_javascript:
- javascript/mathjax.js
- https://polyfill.io/v3/polyfill.min.js?features=es6
Expand Down
35 changes: 32 additions & 3 deletions docs/poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions docs/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,11 @@ packages = [
python = "^3.11"
mkdocs = "^1.5.3"
mkdocs-glightbox = "^0.3.7"
mkdocs-literate-nav = "^0.6.1"
mkdocs-material = "^9.5.12"
mkdocs-same-dir = "^0.1.3"
pygments = "^2.17.2"
mkdocs-autorefs = "^1.0.1"

[tool.poetry.plugins."pygments.lexers"]
sds = "safe_ds_lexer:SafeDsLexer"
Expand Down
7 changes: 0 additions & 7 deletions docs/stdlib/README.md

This file was deleted.

73 changes: 73 additions & 0 deletions docs/stdlib/SUMMARY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
- safeds
- data
- image
- containers
- [Image](safeds/data/image/containers/Image.md)
- tabular
- containers
- [Column](safeds/data/tabular/containers/Column.md)
- [Row](safeds/data/tabular/containers/Row.md)
- [Table](safeds/data/tabular/containers/Table.md)
- [TaggedTable](safeds/data/tabular/containers/TaggedTable.md)
- [TimeSeries](safeds/data/tabular/containers/TimeSeries.md)
- transformation
- [Discretizer](safeds/data/tabular/transformation/Discretizer.md)
- [Imputer](safeds/data/tabular/transformation/Imputer.md)
- [InvertibleTableTransformer](safeds/data/tabular/transformation/InvertibleTableTransformer.md)
- [LabelEncoder](safeds/data/tabular/transformation/LabelEncoder.md)
- [OneHotEncoder](safeds/data/tabular/transformation/OneHotEncoder.md)
- [RangeScaler](safeds/data/tabular/transformation/RangeScaler.md)
- [StandardScaler](safeds/data/tabular/transformation/StandardScaler.md)
- [TableTransformer](safeds/data/tabular/transformation/TableTransformer.md)
- typing
- [ColumnType](safeds/data/tabular/typing/ColumnType.md)
- [Schema](safeds/data/tabular/typing/Schema.md)
- lang
- [AnnotationTarget](safeds/lang/AnnotationTarget.md)
- [Any](safeds/lang/Any.md)
- [Boolean](safeds/lang/Boolean.md)
- [Deprecated](safeds/lang/Deprecated.md)
- [Experimental](safeds/lang/Experimental.md)
- [Expert](safeds/lang/Expert.md)
- [Float](safeds/lang/Float.md)
- [Impure](safeds/lang/Impure.md)
- [ImpurityReason](safeds/lang/ImpurityReason.md)
- [Int](safeds/lang/Int.md)
- [List](safeds/lang/List.md)
- [Map](safeds/lang/Map.md)
- [Nothing](safeds/lang/Nothing.md)
- [Number](safeds/lang/Number.md)
- [Pure](safeds/lang/Pure.md)
- [PythonCall](safeds/lang/PythonCall.md)
- [PythonModule](safeds/lang/PythonModule.md)
- [PythonName](safeds/lang/PythonName.md)
- [Repeatable](safeds/lang/Repeatable.md)
- [String](safeds/lang/String.md)
- [Target](safeds/lang/Target.md)
- ml
- classical
- classification
- [AdaBoostClassifier](safeds/ml/classical/classification/AdaBoostClassifier.md)
- [Classifier](safeds/ml/classical/classification/Classifier.md)
- [DecisionTreeClassifier](safeds/ml/classical/classification/DecisionTreeClassifier.md)
- [GradientBoostingClassifier](safeds/ml/classical/classification/GradientBoostingClassifier.md)
- [KNearestNeighborsClassifier](safeds/ml/classical/classification/KNearestNeighborsClassifier.md)
- [LogisticRegressionClassifier](safeds/ml/classical/classification/LogisticRegressionClassifier.md)
- [RandomForestClassifier](safeds/ml/classical/classification/RandomForestClassifier.md)
- [SupportVectorMachineClassifier](safeds/ml/classical/classification/SupportVectorMachineClassifier.md)
- regression
- [AdaBoostRegressor](safeds/ml/classical/regression/AdaBoostRegressor.md)
- [DecisionTreeRegressor](safeds/ml/classical/regression/DecisionTreeRegressor.md)
- [ElasticNetRegressor](safeds/ml/classical/regression/ElasticNetRegressor.md)
- [GradientBoostingRegressor](safeds/ml/classical/regression/GradientBoostingRegressor.md)
- [KNearestNeighborsRegressor](safeds/ml/classical/regression/KNearestNeighborsRegressor.md)
- [LassoRegressor](safeds/ml/classical/regression/LassoRegressor.md)
- [LinearRegressionRegressor](safeds/ml/classical/regression/LinearRegressionRegressor.md)
- [RandomForestRegressor](safeds/ml/classical/regression/RandomForestRegressor.md)
- [Regressor](safeds/ml/classical/regression/Regressor.md)
- [RidgeRegressor](safeds/ml/classical/regression/RidgeRegressor.md)
- [SupportVectorMachineRegressor](safeds/ml/classical/regression/SupportVectorMachineRegressor.md)
- nn
- [FNNLayer](safeds/ml/nn/FNNLayer.md)
- [NeuralNetworkClassifier](safeds/ml/nn/NeuralNetworkClassifier.md)
- [NeuralNetworkRegressor](safeds/ml/nn/NeuralNetworkRegressor.md)
Loading