Skip to content

general notes

Jump to bottom
Yi Xing edited this page Mar 18, 2024 · 15 revisions

General usage notes

Setting up

Linux/MacOS

  1. Make sure you have Python >= 3.10 installed.
  2. Create a virtual environment if you like.
  3. Download SilverDict.zip from the release page and extract it.
  4. Install the requirements in server/requirements.txt either with pip or your distribution's package manager.
  5. Optional: install the development package of Hunspell, then install the Python bindings with pip if you want to use Hunspell dictionaries. Read more about it below.
  6. Optional: install OpenCC's shared library and the Python package if you need the feature of converting between Simplified and Traditional Chinese. Read more about it below.
  7. Optional: install Xapian and the Python bindings to use full-text search. See below.
  8. Refer to the section below to see how to start the server.
  9. Open the web interface at in your browser, navigate to Library -> Sources, and you would see a default dictionary source. You can add more or just put all dictionaries in the default source directory. Scanning is recursive.
  10. Click on the 'Refresh' button to let SilverDict find and index your dictionaries automatically. If you have a lot of dictionaries, it may take a while. Alternatively, you can check the progress in the command line.
  11. Optional: give the dictionaries some nice aliases in the Dictionaries tab.
  12. Optional: group the dictionaries in the Groups tab.

Windows

A complete Python environment with all the dependencies is bundled with the server inside SilverDict-windows.zip. Read the instructions (README.txt) in the archive carefully.

Starting the server

Linux/MacOS

python server/server.py

The command will start the dictionary server on port 2628. It is the same as dictd, but the protocol is plain HTTP rather than DICT. It also takes an optional argument to specify the address and port to listen on, e.g. python3.10 server/server.py 0.0.0.0 to listen on all interfaces at port 2628, or python3.10 server/server.py 192.168.1.1:80 to listen on port 80 of the interface with the IP address (to serve the LAN). You can also put the address in the configuration file ~/.silverdict/preferences.yaml.

Windows

Double-click the two batch files generated by the setup script, and double-click the .url shortcut to open the web UI in the browser.

Update

To avoid clutter, first remove the subdirectory server/build, then download and extract the new release archive. It is safe to replace all files.

Some important usage notes:

  • The wildcard characters are ^ and + (instead of % and _ of SQL or the more traditional * and ?) for technical reasons. Hint: imagine % and _ are shifted one key to the right on an American keyboard.
  • This project creates a back-up of DSL dictionaries, overhauls (see the bottom of the page for details) them and silently overwrites the original files. So after adding a DSL dictionary to SilverDict, it may no longer work with GoldenDict.
  • During the indexing process of DSL dictionaries, the memory usage could reach as high as 1.5 GiB (tested with the largest DSL ever seen, the Encyclopædia Britannica), and even after that the memory used remains at around 500 MiB. Restart the server process and the memory usage will drop to a few MiB. (The base server with no dictionaries loaded uses around 50 MiB of memory.)

Enabling additional features

In-app updater

SilverDict comes with an in-app updater that connects to GitHub to check for new releases and automatically downloads the archives when the server is started. This could be annoying and is disabled by default. To enable it, you need to edit .silverdict/preferences.yaml, and set check_for_updates to true.

Loading dictionary files into memory

SilverDict cannot make use of the full power of your processors because of the GIL. To speed up the search, you could try loading some dictionary files into memory: create a group named 'Memory' without quotes (the languages do not matter), and add the dictionaries to it. Next time the server will load them into the memory directly. You do not have to query against the Memory group to take advantage to this feature.

The improvement varies. I myself could only observe a 10% speedup, but one user reported a 40% speedup on Android.

Both-sides suggestion matching

This feature is disabled by default. When enabled, suggestions for 'help', for example, would include both 'help', 'helpful', etc. and 'whelp', 'unhelpful', etc. In order to enable it, you need to edit .silverdict/preferences.yaml, which is well documented. Then you should create the ngram index by clicking on the relevant button in the settings dialogue in the web UI. It may take a while.

Each time a dictionary is added or deleted, you need to manually update the index.

If the creation speed is too slow and you are willing to trade space for time, edit .silverdict/preferences.yaml and set ngram_stores_keys to true.

StarDict .syn files support

Support for these files is disabled by default, because they are often completely useless. Here's a sample:

'soudanien': ['soudanienne', 'soudaniennes', 'soudaniens'], 'soudure': ['soudures'], 'soufflage': ['soufflages'], 'soufflant': ['soufflante', 'soufflantes', 'soufflants'], 'soufflard': ['soufflards'], 'soufflerie': ['souffleries'], 'souffle': ['souffles'], 'soufflet': ['soufflets'], 'soufflette': ['soufflettes'], 'souffleur': ['souffleurs', 'souffleuse', 'souffleuses'], 'soufflure': ['soufflures'], 'soufflé': ['soufflée', 'soufflées', 'soufflés'], 'souffrance': ['souffrances'], 'souffrant': ['souffrante', 'souffrantes', 'souffrants'], 'souffreteux': ['souffreteuse', 'souffreteuses'], 'souffre-douleur': ['souffrir douleur'], 'souhaitable': ['souhaitables'], 'souhait': ['souhaits'], 'souillard': ['souillards'], 'souille': ['souilles'], 'souillure': ['souillures'], 'soulagement': ['soulagements'], 'soulte': ['soultes'], 'soulèvement': ['soulèvements'], 'soumission': ['soumissions'], 'soupape': ['soupapes']

If you would like to enable it, edit .silverdict/preferences.yaml and set stardict_load_syns to true.

Morphology analysis

Morphology analysis is implemented with Hunspell, like GoldenDict. You need to install the development package of Hunspell (libhunspell-dev on Debian), then install either of the two Python packages listed below.

Sibel (recommended)

Sibel is developed by myself as a faster and easier to use alternative to the old, unmaintained PyHunspell. In addition to libhunspell, it also uses libicu so you need to have libicu-dev installed to build it. Please refer to the 'Installation' section of its Readme to see how to install it.

PyHunspell

SilverDict also works with PyHunspell. Install it with pip install hunspell or your distro's package manager.

Hunspell dictionaries

Myspell/Hunspell dictionaries are usually distributed as packages on Linux (under /usr/share/hunspell) and MacOS. Sources of additional dictionaries can be found in here. SilverDict looks for Hunspell dictionaries in ~/.silverdict/hunspell, and you should link/copy the .aff/.dic files there. The filename should be in the format of [ISO 639-1 language code].aff/dic, e.g. en.aff & en.dic, en_CA.aff will be ignored.

For morphology features (word stemming and spelling suggestions) to work, you must add the corresponding language code to a group's language list, then all queries against the group will be processed by Hunspell.

Transliteration

Like morphology analysis, transliteration only works for groups with their language field properly set. For example, to search for 'ἀγαθός' with 'agaqos', you need to create a group with your Greek lexicons, and set the language to el.

Greek

The transliteration scheme is Beta Code. No dependency is needed to enable Greek transliteration.

Chinese

Chinese transliteration (简繁转换) is implemented with the library OpenCC. To enable this feature, you should not only set the group's languages, but also install the shared library and data packages together with the Python package opencc. It has been already properly set up in the Windows version.

Unlike Greek, you can also transliterate the definition text. Set your preference in ~/.silverdict/preferences.yaml, which is well documented.

Arabic

The transliteration scheme used is Buckwalter transliteration, with which you can type in ASCII characters to search for Arabic words without diacritics. Like Greek, no dependency is needed to enable Arabic transliteration. Just create a group with your Arabic dictionaries, and set the language to ar.

Custom transformation scripts

You could write custom Python scripts for certain dictionaries to transform the generated HTML. They should be put inside server/app/transformation. The only API requirement is that you should provide a function that takes a string and returns a string, then register it in __init__.py. There is an example script server/app/transformation/michaelis.py for the Michaelis Portuguese dictionaries.

Full-text search

SilverDict uses the powerful Xapian engine as the back-end. To make it work, install libxapian and the Python bindings (libxapian30 and python3-xapian on Debian). lxml is used to convert HTML articles to plain text so it is also strongly recommended.

To get started, create a group named 'Xapian' (without quotes), and add the dictionaries you'd like to be indexed to it. Then go to Settings and create the full-text search index. If you need diacritic-insensitive search, edit preferences.yaml and set full_text_search_diacritic_insensitive to true.

Configuration without the web UI

The configurations are in ~/.silverdict and all are in the human-readable YAML format.

  • dictionaries.yaml is the dictionary list, and you can edit the display names or reorder the dictionaries here, but do not touch dictionary_name, which is used to identify the dictionaries internally.
  • groups.yaml is the group list. The format may not be clear at the first sight, so here is an example:
- lang: !!set
    zh: null
  name: Default Group
- lang: !!set
    en: null
    zh: null
  name: English Dictionaries
- lang: !!set
    en: null
    fr: null
    zh: null
  name: French Dictionaries
- lang: !!set
    en: null
  name: Encyclopaedias
- lang: !!set
    el: null
  name: Greek Lexicons
- lang: !!set
    en: null
    pt: null
    zh: null
  name: Portuguese Dictionaries
  • junction_table.yaml defines the many-to-many relationships between dictionaries and groups. More specifically, it defines which groups each dictionary belongs to. If you change a group's name in groups.yaml, the change has to be made here as well.
  • misc.yaml defines miscellaneous settings:
    • History size: a non-negative integer;
    • Suggestion size: a positive integer;
    • Sources: directories that could be scanned for new dictionaries automatically;
  • preferences.yaml is a commented configuration file that keeps options that are not meant to be changed at runtime.

To avoid performance problems, it is advised to keep history/suggestion size small. Do not delete the default group!

* What it does: (1) decompress the dictionary file if compressed; (2) remove the BOM, non-printing characters and strange symbols (only {·} currently) from the text; (3) normalize the initial whitespace characters of definition lines; (4) overwrite the .dsl file with UTF-8 encoding and re-compress with dictzip. After this process the file is smaller and easier to work with.

Clone this wiki locally