LIT documentation update post demo cleanup.

PiperOrigin-RevId: 646541561

website/sphinx_src/api.md

@@ -1,6 +1,6 @@
 # LIT Python API
 
-<!--* freshness: { owner: 'lit-dev' reviewed: '2023-08-23' } *-->
+<!--* freshness: { owner: 'lit-dev' reviewed: '2024-06-24' } *-->
 
 <!-- [TOC] placeholder - DO NOT REMOVE -->
 
@@ -349,7 +349,7 @@ list of scores for each token. The Integrated Gradients saliency method
 additionally requires a `TokenEmbeddings` input and corresponding output, as
 well as a label field `Target` to pin the gradient target to the same class as
 an input and corresponding output. See the
-[GLUE models class](https://github.com/PAIR-code/lit/blob/main/lit_nlp/examples/models/glue_models.py)
+[GLUE models class](https://github.com/PAIR-code/lit/blob/main/lit_nlp/examples/glue/models.py)
 for an example of these spec requirements.
 
 The core API involves implementing the `run()` method:
@@ -712,11 +712,9 @@ this can cause jitter (UI modules appearing, disappearing, reordering, resizing,
 etc.) when switching between models or datasets with heterogeneous `Spec`s.
 
 When implementing your own LIT components and modules, you can use
-[`utils.find_spec_keys()`][utils-lib]
-(Python) and
-[`findSpecKeys()`][utils-lib]
-(TypeScript) to identify fields of interest in a `Spec`. These methods recognize
-and respect subclasses. For example,
+[`utils.find_spec_keys()`][utils-lib-py] (Python) and
+[`findSpecKeys()`][utils-lib] (TypeScript) to identify fields of interest in a
+`Spec`. These methods recognize and respect subclasses. For example,
 `utils.find_spec_keys(spec, Scalar)` will also match any `RegressionScore`
 fields, but `utils.find_spec_keys(spec, RegressionScore)` will not return all
 `Scalar` fields in the `Spec`.
@@ -807,8 +805,12 @@ _See the [examples](https://github.com/PAIR-code/lit/blob/main/lit_nlp/examples)
 
 ### Available types
 
-The full set of `LitType`s is defined in [types.py](https://github.com/PAIR-code/lit/blob/main/lit_nlp/api/types.py). Numeric types such as `Integer` and `Scalar` have predefined ranges that can be overridden using corresponding `min_val` and `max_val` attributes as seen [here](https://github.com/PAIR-code/lit/blob/main/lit_nlp/examples/datasets/penguin_data.py;l=19-22;rcl=574999438). The different types available in LIT are summarized
-in the table below.
+The full set of `LitType`s is defined in
+[types.py](https://github.com/PAIR-code/lit/blob/main/lit_nlp/api/types.py). Numeric types
+such as `Integer` and `Scalar` have predefined ranges that can be overridden
+using corresponding `min_val` and `max_val` attributes as seen
+[here](https://github.com/PAIR-code/lit/blob/main/lit_nlp/examples/penguin/data.py;l=19-22;rcl=639554825).
+The different types available in LIT are summarized in the table below.
 
 Note: Bracket syntax, such as `<float>[num_tokens]`, refers to the shapes of
 NumPy arrays where each element inside the brackets is an integer.
@@ -981,15 +983,15 @@ needing to reload the server or click the UI.
 For example, to view examples in a dataset:
 
 ```python
-from lit_nlp.examples.datasets import glue
-dataset = glue.SST2Data('validation')
+from lit_nlp.examples.glue import data as glue_data
+dataset = glue_data.SST2Data('validation')
 print(dataset.examples)  # list of records {"sentence": ..., "label": ...}
 ```
 
 And to run inference on a few of them:
 
 ```python
-from lit_nlp.examples.models import glue_models
+from lit_nlp.examples.glue import models as glue_models
 
 model = glue_models.SST2Model("/path/to/model/files")
 preds = list(model.predict(dataset.examples[:5]))
@@ -1013,16 +1015,19 @@ For a full working example in Colab, see [LIT_components_example.ipynb](https://
 <!-- Links -->
 
 [build-metadata]: https://github.com/PAIR-code/lit/blob/main/lit_nlp/app.py
-[components-py]: https://github.com/PAIR-code/lit/blob/main/lit_nlp/api/dataset.py
+[components-py]: https://github.com/PAIR-code/lit/blob/main/lit_nlp/api/components.py
 [curves-interp]: https://github.com/PAIR-code/lit/blob/main/lit_nlp/components/curves.py
 [dataset-py]: https://github.com/PAIR-code/lit/blob/main/lit_nlp/api/dataset.py
 [grad-maps]: https://github.com/PAIR-code/lit/blob/main/lit_nlp/components/gradient_maps.py
 [json]: https://www.json.org
 [mnli-dataset]: https://cims.nyu.edu/~sbowman/multinli/
+
 [mnli-demo]: https://pair-code.github.io/lit/demos/glue.html
-[model-py]: https://github.com/PAIR-code/lit/blob/main/lit_nlp/api/dataset.py
+
+[model-py]: https://github.com/PAIR-code/lit/blob/main/lit_nlp/api/model.py
 [should_display_module]: https://github.com/PAIR-code/lit/blob/main/lit_nlp/client/core/lit_module.ts
 [types_py]: https://github.com/PAIR-code/lit/blob/main/lit_nlp/api/types.py
 [types_ts]: https://github.com/PAIR-code/lit/blob/main/lit_nlp/client/lib/lit_types.ts
 [utils-lib]: https://github.com/PAIR-code/lit/blob/main/lit_nlp/client/lib/utils.ts
+[utils-lib-py]: https://github.com/PAIR-code/lit/blob/main/lit_nlp/lib/utils.py
 [word-replacer]: https://github.com/PAIR-code/lit/blob/main/lit_nlp/components/word_replacer.py

website/sphinx_src/components.md

@@ -1,8 +1,6 @@
 # Components and Features
 
-<!--* freshness: { owner: 'lit-dev' reviewed: '2024-02-20' } *-->
-
-TODO(b/343678324): Remove the files after cl/629748253 is submitted.
+<!--* freshness: { owner: 'lit-dev' reviewed: '2024-06-24' } *-->
 
 <!-- [TOC] placeholder - DO NOT REMOVE -->
 
@@ -258,7 +256,7 @@ regression (`RegressionScore`) and generation (`GeneratedText` or
 ### Gradient Norm
 
 This is a simple method, in which salience scores are proportional to the L2
-norm of the gradient, i.e. the score for token $i$ is:
+norm of the gradient, i.e. the score for token $$i$$ is:
 
 $$S(i) \propto ||\nabla_{x_i} \hat{y}||_2$$
 
@@ -270,40 +268,40 @@ To enable this method, your model should, as part of the
 *   Return a `TokenGradients` field with the `align` attribute pointing to the
     name of the `Tokens` field (i.e. `align="tokens"`). Values should be arrays
     of shape `<float>[num_tokens, emb_dim]` representing the gradient
-    $\nabla_{x} \hat{y}$ of the embeddings with respect to the prediction
-    $\hat{y}$.
+    $$\nabla_{x} \hat{y}$$ of the embeddings with respect to the prediction
+    $$\hat{y}$$.
 
 Because LIT is framework-agnostic, the model code is responsible for performing
 the gradient computation and returning the result as a NumPy array. The choice
-of $\hat{y}$ is up to the developer; typically for regression/scoring this is
+of $$\hat{y}$$ is up to the developer; typically for regression/scoring this is
 the raw score and for classification this is the score of the predicted (argmax)
 class.
 
 ### Gradient-dot-Input
 
 In this method, salience scores are proportional to the dot product of the input
-embeddings and their gradients, i.e. for token $i$ we compute:
+embeddings and their gradients, i.e. for token $$i$$ we compute:
 
 $$S(i) \propto x_i \cdot \nabla_{x_i} \hat{y}$$
 
 Compared to grad-norm, this gives directional scores: a positive score is can be
 interpreted as that token having a positive influence on the prediction
-$\hat{y}$, while a negative score suggests that the prediction would be stronger
-if that token was removed.
+$$\hat{y}$$, while a negative score suggests that the prediction would be
+stronger if that token was removed.
 
 To enable this method, your model should, as part of the
 [output spec and `predict()` implementation](./api.md#models):
 
 *   Return a `Tokens` field with values (as `list[str]`) containing the
     tokenized input.
 *   Return a `TokenEmbeddings` field with values as arrays of shape
-    `<float>[num_tokens, emb_dim]` containing the input embeddings $x$.
+    `<float>[num_tokens, emb_dim]` containing the input embeddings $$x$$.
 *   Return a `TokenGradients` field with the `align` attribute pointing to the
     name of the `Tokens` field (i.e. `align="tokens"`), and the `grad_for`
     attribute pointing to the name of the `TokenEmbeddings` field. Values should
     be arrays of shape `<float>[num_tokens, emb_dim]` representing the gradient
-    $\nabla_{x} \hat{y}$ of the embeddings with respect to the prediction
-    $\hat{y}$.
+    $$\nabla_{x} \hat{y}$$ of the embeddings with respect to the prediction
+    $$\hat{y}$$.
 
 As with grad-norm, the model should return embeddings and gradients as NumPy
 arrays. The LIT `GradientDotInput` component will compute the dot products and
@@ -525,9 +523,7 @@ your model should, as part of the
     input image.
 
     The model should be able to accept input images as numpy arrays in addition
-    to accepting base64 URL encoded format. See
-    [mobilenet.py](https://github.com/PAIR-code/lit/blob/main/lit_nlp/examples/mobilenet.py)
-    for an example.
+    to accepting base64 URL encoded format.
 
 A variety of image saliency techniques are implemented for models that return
 image gradients, through use of the
@@ -538,36 +534,6 @@ and XRAI.
 Each of these techniques returns a saliency map image as a base64-encoded string
 through the `ImageSalience` type.
 
-## Attention
-
-LIT can display a visualization of attention heads from transformers and other
-models:
-
-![Attention Visualization](./images/components/attention.png){w=400px align=center}
-
-To enable this, your model should return one or more fields of the type
-`AttentionHeads`, with values as arrays of shape `<float>[num_heads, num_tokens,
-num_tokens]`. Each field represents a set of heads at a single layer of the
-model, so models will often have more than one:
-
-```python
-   def output_spec(self) -> types.Spec:
-     return {
-         # ...
-         "tokens": lit_types.Tokens(parent="input_text"),
-         "layer_0/attention": lit_types.AttentionHeads(align_in="tokens", align_out="tokens"),
-         "layer_1/attention": lit_types.AttentionHeads(align_in="tokens", align_out="tokens"),
-         "layer_2/attention": lit_types.AttentionHeads(align_in="tokens", align_out="tokens"),
-         # ...
-     }
-```
-
-The `align_in` and `align_out` attributes are the names of `Tokens` fields for
-the source and target tokens for that layer. For self-attention (such as in
-BERT) these would name the same field, but for encoder-decoder attention they
-could reference different token sets (such as "input_tokens" and
-"target_tokens").
-
 ## Embedding Projector
 
 LIT includes a version of the
@@ -599,7 +565,7 @@ example, we could facet by class label:
 
 ![Metrics Table](./images/components/metrics-table.png)
 
-To try this out, see https://pair-code.github.io/lit/demos/glue.html and navigate to the "Performance" tab.
+To try this out, see https://pair-code.github.io/lit/demos/glue.html and navigate to the "Metrics" tab.
 
 To enable metrics, your model should set the `parent` attribute on one or more
 output fields, pointing to the name of the input field that it should be
@@ -638,7 +604,7 @@ analysis.
 
 ![Confusion Matrix](./images/components/confusion-matrix.png){w=600px align=center}
 
-To try this out, see https://pair-code.github.io/lit/demos/glue.html and navigate to the "Performance" tab.
+To try this out, see https://pair-code.github.io/lit/demos/glue.html and navigate to the "Metrics" tab.
 
 The confusion matrix is supported for classification models, or if the input
 data includes any categorical features (`CategoryLabel`).
@@ -724,6 +690,8 @@ datapoints, giving a global view of feature effects.
 
 ![Partial Dependence Plots Module](./images/components/lit-pdps.png){w=400px align=center}
 
+To try this out, see https://pair-code.github.io/lit/demos/penguins.html and navigate to the "Predictions" tab.
+
 ### Dive
 
 Dive is a visualization module, inspired by our prior work on

website/sphinx_src/demos.md

@@ -1,6 +1,6 @@
 # Demos
 
-<!-- freshness: { owner: 'lit-dev' reviewed: '2023-08-29' } -->
+<!-- freshness: { owner: 'lit-dev' reviewed: '2024-06-24' } -->
 
 <!-- [TOC] placeholder - DO NOT REMOVE -->
 
@@ -36,7 +36,6 @@ https://pair-code.github.io/lit/demos/.
         the margin.
     *   Embedding projector to find clusters in representation space.
     *   Integrated Gradients, LIME, and other salience methods.
-    *   Attention visualization.
     *   Counterfactual generators, including HotFlip for targeted adversarial
         perturbations.
 
@@ -61,7 +60,6 @@ https://pair-code.github.io/lit/tutorials/sentiment
         quick filtering.
     *   Embedding projector to find clusters in representation space.
     *   Integrated Gradients, LIME, and other salience methods.
-    *   Attention visualization.
 
 --------------------------------------------------------------------------------
 
@@ -72,22 +70,19 @@ https://pair-code.github.io/lit/tutorials/sentiment
 **Code:**
 [examples/prompt_debugging/server.py](https://github.com/PAIR-code/lit/blob/main/lit_nlp/examples/prompt_debugging/server.py)
 
-*   Supports Gemma 2B and 7B models using KerasNLP and TensorFlow.
+*   Supports Gemma 2B and 7B models using KerasNLP (with TensorFlow or PyTorch)
+    and Transformers (with PyTorch).
 *   Interactively debug LLM prompts using
     [sequence salience](./components.md#sequence-salience).
 *   Multiple salience methods (grad-l2 and grad-dot-input), at multiple
-    granularities: token-, word-, sentence-, and paragraph-level.
+    granularities: token-, word-, line-, sentence-, and paragraph-level.
 
 Tip: check out the in-depth walkthrough at
 https://ai.google.dev/responsible/model_behavior, part of the Responsible
 Generative AI Toolkit.
 
 --------------------------------------------------------------------------------
 
-## Structured Prediction <!-- DO NOT REMOVE {#structured .demo-section-header} -->
-
---------------------------------------------------------------------------------
-
 ## Multimodal <!-- DO NOT REMOVE {#multimodal .demo-section-header} -->
 
 ### Tabular Data: Penguin Classification <!-- DO NOT REMOVE {#penguin .demo-header} -->

website/sphinx_src/frontend_development.md

@@ -1,6 +1,6 @@
 # Frontend Developer Guide
 
-<!--* freshness: { owner: 'lit-dev' reviewed: '2023-08-23' } *-->
+<!--* freshness: { owner: 'lit-dev' reviewed: '2024-06-24' } *-->
 
 <!-- [TOC] placeholder - DO NOT REMOVE -->
 
@@ -46,9 +46,9 @@ and rendering the various `LitModule` components, a process about which we'll go
 into greater detail later.
 
 The JS bundle entry point is
-[`main.ts`](https://github.com/PAIR-code/lit/blob/main/lit_nlp/client/default/main.ts), which
-first imports the loaded, the `<lit-app>` web component is declared, and
-attaches itself to the DOM, waiting for the app to be initialized.
+[`main.ts`](https://github.com/PAIR-code/lit/blob/main/lit_nlp/client/main.ts), which first
+imports the loaded, the `<lit-app>` web component is declared, and attaches
+itself to the DOM, waiting for the app to be initialized.
 
 The second step is kicking off app initialization. The
 [`LitApp`](https://github.com/PAIR-code/lit/blob/main/lit_nlp/client/core/app.ts) singleton
@@ -67,7 +67,7 @@ pre-configured layouts in
 *   `default`: The original LIT layout with a single group of modules on top for
     exploring and selecting data, and a collection of tabs supporting different
     analytical tasks on the bottom; and
-*   `experimental`: A three-panel layout that puts exploratory data
+*   `three_panel`: A three-panel layout that puts exploratory data
     visualizations at full-page height on the left, tools for inspecting and
     manipulating examples and their associated predictions in the upper right,
     and a collection of tabs supporting different analytical tasks in the lower
@@ -84,7 +84,7 @@ Note: The pre-configured layouts are added to every `LitApp` instance using
 the Mapping passed to the `LitApp` constructor overrides the pre-configured
 layouts `Mapping`. Thus, you can remove or change these pre-configured layouts
 as you like by passing a `Mapping` where the values of `simple`, `default`,
-and/or `experimental` is `None` (to remove) or a `LitCanonicalLayout` instance
+and/or `three_panel` is `None` (to remove) or a `LitCanonicalLayout` instance
 (to override) as you desire.
 
 The actual layout of components in the LIT UI, see

website/sphinx_src/getting_started.md

@@ -1,6 +1,6 @@
 # Getting Started with LIT
 
-<!--* freshness: { owner: 'lit-dev' reviewed: '2022-08-17' } *-->
+<!--* freshness: { owner: 'lit-dev' reviewed: '2024-06-25' } *-->
 
 <!-- [TOC] placeholder - DO NOT REMOVE -->