refactor: overhaul build and CI workflows

- Simplified environment setup by integrating Rye for dependency management. - Streamlined CI pipeline by removing redundant steps and optimizing for efficiency. - Enhanced code quality by updating Python dependencies and targeting Python 3.12. - Improved documentation and README links for better user guidance. - Optimized rule set compilation and artifact handling in CI workflows. - Removed deprecated files and configurations to clean up the codebase. - Introduced lazy loading for modules to improve startup performance. - Refactored logging initialization for better consistency and readability. - Updated project metadata and configuration files for modern standards. BREAKING CHANGE: Dropped support for Node 6 and older Python versions.
liblaf · Sep 5, 2024 · aceb22b · aceb22b
1 parent 3e0e2e9
commit aceb22b
Show file tree

Hide file tree

Showing 71 changed files with 1,108 additions and 1,020 deletions.
diff --git a/.cspell.json b/.cspell.json
@@ -1,17 +1,22 @@
 {
   "words": [
+    "aiter",
+    "anyio",
     "asyncio",
-    "bidi",
-    "commitizen",
-    "dtemp",
+    "dpath",
     "eifinger",
     "emby",
     "geoip",
+    "gstatic",
     "httpx",
+    "levelno",
     "liblaf",
+    "optim",
     "pycache",
     "pydantic",
     "pydocstyle",
+    "rcode",
+    "ubelt",
     "venv"
   ],
   "ignorePaths": ["**/*-lock.*", "**/*.lock*", "**/.cspell.json"],

diff --git a/.envrc b/.envrc
@@ -1,2 +1,6 @@
 # shellcheck disable=SC2148
-source_env_if_exists ./.venv/bin/activate
+watch_file requirements{,-dev}.lock
+if [[ ! -f .venv/bin/activate ]]; then
+  rye sync
+fi
+source_env .venv/bin/activate
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
@@ -15,24 +15,20 @@ jobs:
       - name: Setup Rye
         uses: eifinger/setup-rye@v4
       - name: Install sing-box
-        # run: bash <(curl -fsSL https://sing-box.app/deb-install.sh)
-        # TODO: upstream issue: <https://github.com/SagerNet/sing-box/issues/1940>
-        run: |-
-          gh release --repo "SagerNet/sing-box" download "v1.10.0-alpha.28" --pattern "sing-box-*-linux-amd64.tar.gz"
-          tar --extract --file sing-box-*-linux-amd64.tar.gz --verbose
-          install -D --no-target-directory --verbose sing-box-*-linux-amd64/sing-box ~/.local/bin/sing-box
+        run: bash <(curl -fsSL https://sing-box.app/deb-install.sh)
         env:
           GH_TOKEN: ${{ github.token }}
-      - name: Install Python Dependencies
+      - name: Install Dependencies
         run: rye sync --no-lock
-      - name: Install Prettier
-        run: npm install --global prettier
       - name: Build Rule Sets
-        run: make --jobs
+        run: |-
+          source .venv/bin/activate
+          python src/build.py
+          npx prettier --write output/README.md
       - name: Upload Artifacts
         uses: actions/upload-artifact@v4
         with:
-          name: rule-sets
+          name: sing
           path: output
 
   deploy:
@@ -47,12 +43,12 @@ jobs:
       - name: Download Artifacts
         uses: actions/download-artifact@v4
         with:
-          name: rule-sets
-          path: output
+          name: sing
+          path: sing
       - name: Deploy to GitHub Branch
         uses: peaceiris/actions-gh-pages@v4
         with:
           github_token: ${{ github.token }}
-          publish_branch: rule-sets
-          publish_dir: output
+          publish_branch: sing
+          publish_dir: sing
           force_orphan: true
diff --git a/.gitignore b/.gitignore
@@ -11,4 +11,3 @@ wheels/
 
 data/
 output/
-rule-sets/
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
diff --git a/.ruff.toml b/.ruff.toml
@@ -1,6 +1,6 @@
 fix = true
 show-fixes = true
-target-version = "py311"
+target-version = "py312"
 
 [format]
 docstring-code-format = true
@@ -16,9 +16,9 @@ ignore = [
   "FIX",
   "INP",
   "ISC",
-  "N806",
   "PLR09",
   "PLR2004",
+  "RET504",
   "S",
   "T20",
   "TD",

diff --git a/Makefile b/Makefile
diff --git a/README.md b/README.md
@@ -1,28 +1,28 @@
 # sing-box Rules
 
-| Name                                       | Download Link                                                                                         |
-| ------------------------------------------ | ----------------------------------------------------------------------------------------------------- |
-| [📵 RuleSet: ADs](#-ruleset-ads)           | [rule-set/ads.srs](https://github.com/liblaf/sing-box-rules/raw/rule-sets/rule-set/ads.srs)           |
-| [🔒 RuleSet: Private](#-ruleset-private)   | [rule-set/private.srs](https://github.com/liblaf/sing-box-rules/raw/rule-sets/rule-set/private.srs)   |
-| [🇨🇳 RuleSet: CN](#-ruleset-cn)             | [rule-set/cn.srs](https://github.com/liblaf/sing-box-rules/raw/rule-sets/rule-set/cn.srs)             |
-| [🌐 RuleSet: Proxy](#-ruleset-proxy)       | [rule-set/proxy.srs](https://github.com/liblaf/sing-box-rules/raw/rule-sets/rule-set/proxy.srs)       |
-| [🤖 RuleSet: AI](#-ruleset-ai)             | [rule-set/ai.srs](https://github.com/liblaf/sing-box-rules/raw/rule-sets/rule-set/ai.srs)             |
-| [🍟 RuleSet: Emby](#-ruleset-emby)         | [rule-set/emby.srs](https://github.com/liblaf/sing-box-rules/raw/rule-sets/rule-set/emby.srs)         |
-| [☁️ RuleSet: Download](#-ruleset-download) | [rule-set/download.srs](https://github.com/liblaf/sing-box-rules/raw/rule-sets/rule-set/download.srs) |
-| [📺 RuleSet: Media](#-ruleset-media)       | [rule-set/media.srs](https://github.com/liblaf/sing-box-rules/raw/rule-sets/rule-set/media.srs)       |
-| [📵 GeoSite: ADs](#-ruleset-ads)           | [geosite/ads.srs](https://github.com/liblaf/sing-box-rules/raw/rule-sets/geosite/ads.srs)             |
-| [🔒 GeoSite: Private](#-ruleset-private)   | [geosite/private.srs](https://github.com/liblaf/sing-box-rules/raw/rule-sets/geosite/private.srs)     |
-| [🇨🇳 GeoSite: CN](#-ruleset-cn)             | [geosite/cn.srs](https://github.com/liblaf/sing-box-rules/raw/rule-sets/geosite/cn.srs)               |
-| [🌐 GeoSite: Proxy](#-ruleset-proxy)       | [geosite/proxy.srs](https://github.com/liblaf/sing-box-rules/raw/rule-sets/geosite/proxy.srs)         |
-| [🇨🇳 GeoIP: CN](#-ruleset-cn)               | [geoip/cn.srs](https://github.com/liblaf/sing-box-rules/raw/rule-sets/geoip/cn.srs)                   |
-
-- [statistics](https://github.com/liblaf/sing-box-rules/blob/rule-sets/README.md)
+| Name                                       | Download Link                                                                                    |
+| ------------------------------------------ | ------------------------------------------------------------------------------------------------ |
+| [📵 RuleSet: ADs](#-ruleset-ads)           | [rule-set/ads.srs](https://github.com/liblaf/sing-box-rules/raw/sing/rule-set/ads.srs)           |
+| [🔒 RuleSet: Private](#-ruleset-private)   | [rule-set/private.srs](https://github.com/liblaf/sing-box-rules/raw/sing/rule-set/private.srs)   |
+| [🇨🇳 RuleSet: CN](#-ruleset-cn)             | [rule-set/cn.srs](https://github.com/liblaf/sing-box-rules/raw/sing/rule-set/cn.srs)             |
+| [🌐 RuleSet: Proxy](#-ruleset-proxy)       | [rule-set/proxy.srs](https://github.com/liblaf/sing-box-rules/raw/sing/rule-set/proxy.srs)       |
+| [🤖 RuleSet: AI](#-ruleset-ai)             | [rule-set/ai.srs](https://github.com/liblaf/sing-box-rules/raw/sing/rule-set/ai.srs)             |
+| [🍟 RuleSet: Emby](#-ruleset-emby)         | [rule-set/emby.srs](https://github.com/liblaf/sing-box-rules/raw/sing/rule-set/emby.srs)         |
+| [☁️ RuleSet: Download](#-ruleset-download) | [rule-set/download.srs](https://github.com/liblaf/sing-box-rules/raw/sing/rule-set/download.srs) |
+| [📺 RuleSet: Media](#-ruleset-media)       | [rule-set/media.srs](https://github.com/liblaf/sing-box-rules/raw/sing/rule-set/media.srs)       |
+| [📵 GeoSite: ADs](#-ruleset-ads)           | [geosite/ads.srs](https://github.com/liblaf/sing-box-rules/raw/sing/geosite/ads.srs)             |
+| [🔒 GeoSite: Private](#-ruleset-private)   | [geosite/private.srs](https://github.com/liblaf/sing-box-rules/raw/sing/geosite/private.srs)     |
+| [🇨🇳 GeoSite: CN](#-ruleset-cn)             | [geosite/cn.srs](https://github.com/liblaf/sing-box-rules/raw/sing/geosite/cn.srs)               |
+| [🌐 GeoSite: Proxy](#-ruleset-proxy)       | [geosite/proxy.srs](https://github.com/liblaf/sing-box-rules/raw/sing/geosite/proxy.srs)         |
+| [🇨🇳 GeoIP: CN](#-ruleset-cn)               | [geoip/cn.srs](https://github.com/liblaf/sing-box-rules/raw/sing/geoip/cn.srs)                   |
+
+- [statistics](https://github.com/liblaf/sing-box-rules/blob/sing/README.md)
 - `GeoSite: *` does not contain `IP-CIDR` rules, useful for DNS Rule.
 - `GeoIP: *` does not contain `DOMAIN*` rules, useful for DNS Rule.
 
 ## Optimization
 
-[optimization results](https://github.com/liblaf/sing-box-rules/blob/rule-sets/README.md)
+[optimization results](https://github.com/liblaf/sing-box-rules/blob/sing/README.md)
 
 - remove duplicate rules
 - merge `DOMAIN` with `DOMAIN-SUFFIX`
@@ -31,7 +31,7 @@
 - merge `DOMAIN-SUFFIX` with `DOMAIN-KEYWORD`
 - merge `IP-CIDR`
 
-## Example [sing-box] Config
+## Example [sing-box](https://sing-box.sagernet.org) Config
 
 ### DNS Rules
 
@@ -44,34 +44,20 @@
         "address": "https://cloudflare-dns.com/dns-query",
         "address_resolver": "dns:bootstrap"
       },
-      {
-        "tag": "dns:cn",
-        "address": "https://dns.alidns.com/dns-query",
-        "address_resolver": "dns:bootstrap"
-      },
-      { "tag": "dns:bootstrap", "address": "223.5.5.5", "detour": "DIRECT" },
       { "tag": "dns:local", "address": "local" },
       { "tag": "dns:reject", "address": "rcode://refused" }
     ],
     "rules": [
-      { "outbound": "any", "server": "dns:bootstrap" },
+      { "outbound": "any", "server": "dns:local" },
       {
         "rule_set": "geosite:ads",
         "server": "dns:reject",
         "disable_cache": true
       },
       { "rule_set": "geosite:private", "server": "dns:local" },
-      { "clash_mode": "direct", "server": "dns:cn" },
+      { "clash_mode": "direct", "server": "dns:local" },
       { "clash_mode": "global", "server": "dns:proxy" },
-      {
-        "type": "logical",
-        "mode": "and",
-        "rules": [
-          { "rule_set": "geosite:proxy", "invert": true },
-          { "rule_set": "geosite:cn" }
-        ],
-        "server": "dns:cn"
-      },
+      { "rule_set": "geosite:cn", "server": "dns:local" },
       {
         "type": "logical",
         "mode": "and",
@@ -184,9 +170,9 @@
   - [DustinWin/geosite-all.db](https://github.com/DustinWin/ruleset_geodata): `proxy`
   - [MetaCubeX/geosite.db](https://github.com/MetaCubeX/meta-rules-dat): `*!cn*`
 - exclude:
+  - [🇨🇳 RuleSet: CN](#-ruleset-cn)
   - [📵 RuleSet: ADs](#-ruleset-ads)
   - [🔒 RuleSet: Private](#-ruleset-private)
-  - [🇨🇳 RuleSet: CN](#-ruleset-cn)
 
 ### 🤖 RuleSet: AI
 
@@ -198,12 +184,18 @@
   - [DustinWin/geosite-all.db](https://github.com/DustinWin/ruleset_geodata): `ai`
   - [MetaCubeX/geosite.db](https://github.com/MetaCubeX/meta-rules-dat): `openai`
 - exclude:
+  - [🇨🇳 RuleSet: CN](#-ruleset-cn)
   - [📵 RuleSet: ADs](#-ruleset-ads)
+  - [🔒 RuleSet: Private](#-ruleset-private)
 
 ### 🍟 RuleSet: Emby
 
 - include:
   - [NotSFC/Emby.json](https://github.com/NotSFC/rulelist/blob/main/sing-box/Emby/Emby.json)
+- exclude:
+  - [🇨🇳 RuleSet: CN](#-ruleset-cn)
+  - [📵 RuleSet: ADs](#-ruleset-ads)
+  - [🔒 RuleSet: Private](#-ruleset-private)
 
 ### ☁️ RuleSet: Download
 
@@ -216,8 +208,9 @@
   - [liblaf/download.json](https://github.com/liblaf/sing-box-rules/blob/main/custom/download.json)
   - [MetaCubeX/geosite.db](https://github.com/MetaCubeX/meta-rules-dat): `category-dev` | `onedrive`
 - exclude:
-  - [📵 RuleSet: ADs](#-ruleset-ads)
   - [🇨🇳 RuleSet: CN](#-ruleset-cn)
+  - [📵 RuleSet: ADs](#-ruleset-ads)
+  - [🔒 RuleSet: Private](#-ruleset-private)
 
 ### 📺 RuleSet: Media
 
@@ -229,8 +222,9 @@
   - [MetaCubeX/geosite-lite.db](https://github.com/MetaCubeX/meta-rules-dat): `proxymedia`, `youtube`
   - [MetaCubeX/geosite.db](https://github.com/MetaCubeX/meta-rules-dat): `youtube`
 - exclude:
-  - [📵 RuleSet: ADs](#-ruleset-ads)
   - [🇨🇳 RuleSet: CN](#-ruleset-cn)
+  - [📵 RuleSet: ADs](#-ruleset-ads)
+  - [🔒 RuleSet: Private](#-ruleset-private)
 
 ## Acknowledgement
 

diff --git a/custom/cn.json b/custom/cn.json
diff --git a/custom/download.json b/custom/download.json
diff --git a/makefiles/data.mk b/makefiles/data.mk