Merge pull request #161 from nzbgetcom/develop

nzbget-23.0 - Features: - Extension Manager [#76](7348b19); * The new nzbget extension system, which makes it easy to download/update/delete extensions with backward compatibility with the old system; * extensions [master list](https://github.com/nzbgetcom/nzbget-extensions/); * changed: - RPC request "configtemplates" - no longer returns script templates, but only the config template; * added: - new RPC requests: - "loadextensions" - loads all extensions from {ScriptDIR} and returns an array of structures in JSON/XML formats; - "updateextension" - downloads by url and name and installs the extension. Returns 'true' or error response in JSON/XML formats; - "deleteextension" - deletes extension by name. Returns 'true' or error response in JSON/XML formats; - "downloadextension" - downloads by url and installs the extension. Returns 'true' or error response in JSON/XML formats; - "testextension" - tries to find the right executor program for the extension, e.g. Python. Returns 'true' or error response in JSON/XML formats; - "EXTENSION MANAGER" section in webui to download/delete/update extensions; - Boost.Json library to work with JSON; - more unit tests; * refactored: - replaced raw pointers with smart pointers and const refs where possible for memory safty reasons; * removed: - testdata_FILES from Makefile.am; - EMail and Logger scripts; - Docker support [#55](1913e33); - Synology support (spk) [#72](3f8fd6d); - QNAP support [#158](9676c77); - aarch64 mipseb mipsel ppc6xx ppc500 architectures to linux build [#61](9599854) [#146](fcc104d); - article read chunk size [#52](1aa42f2); * Added ArticleReadChunkSize config option which allows to adjust the buffer size for customization on different platforms, which can lead to increased performance; - increased the number of default connections to 8 [#54](55eca4c); - automatic search for a suitable interpreter on POSIX [#74](ec0756d); - certificate verification levels [#150](5956a17); * levels: - None: NO certificate signing check, NO certificate hostname check; - Minimal: certificate signing check, NO certificate hostname check; - Strict: certificate signing check, certificate hostname check; * tested on a mock nzbget NNTP nserv server with self-signed certificate and got expected results: - "Strict" -> test failed; - "Minimal" -> test failed; - "None" -> test passed"; - Bug fixies: - possible memory corruption [#148](16ab25d); - For developers: - fixed unit tests (Windows only for now) and started migration to CMake [#64](73e8c00); * We are going to completely migrate to CMake as a more universal one for cross-platform development and drop autotools and MSBuild; - using libxml2 on Windows and vcpkg package manager to install dependencies [#70](f8cb2bd); * libxml2 library is now used on Windows to work with xml in the same way we already do on Linux and macOS; * removed platform-dependent code for working with xml on Windows;
nzbgetcom · Feb 9, 2024 · c116a63 · c116a63
2 parents de21f8a + 4477908
commit c116a63
Show file tree

Hide file tree

Showing 211 changed files with 9,093 additions and 30,976 deletions.
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,11 @@
+## Description
+
+Brief explanation of the goal, link issues covered (partially or fully resolved) by this pull request
+
+## Lib changes
+
+If vendored libraries are getting changed, please list new version, old version, and what kind of dependencies or restrictions it may introduce
+
+## Testing
+
+Mention smoke testing done or extra QA instructions
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
@@ -0,0 +1,141 @@
+name: build
+
+on:
+  push:
+    branches:
+    - develop
+    - main
+  workflow_dispatch:
+
+jobs:
+  build-windows:
+    uses: ./.github/workflows/windows.yml
+
+  build-linux:
+    uses: ./.github/workflows/linux.yml
+
+  build-osx:
+    uses: ./.github/workflows/osx.yml
+
+  build-synology:
+    uses: ./.github/workflows/synology.yml
+
+  build-qnap:
+    uses: ./.github/workflows/qnap.yml
+
+  repack-qnap:
+    uses: ./.github/workflows/qnap-repack.yml
+    with:
+      external_call: true
+    needs: [build-linux]
+    permissions:
+      actions: write
+
+  generate-signatures:
+    env:
+      PRIVATE_KEY: ${{ secrets.PRIVATE_KEY }}
+    runs-on: ubuntu-latest
+    needs: [build-windows, build-linux, build-osx, build-synology, build-qnap, repack-qnap]
+    permissions:
+      actions: write
+    steps:
+
+    - name: Download build artifacts
+      uses: actions/download-artifact@v4
+
+    - name: Generate signatures
+      run: |
+        mkdir -p builds
+        mv nzbget-windows-installers/* builds || true
+        mv nzbget-linux-installers/* builds || true
+        mv nzbget-osx-installers/* builds || true
+        mv nzbget-synology-packages/* builds || true
+        mv nzbget-qnap-packages/* builds || true
+        mv nzbget-qnap-native-packages/* builds || true
+        cd builds
+        VERSION=$(ls | grep bin-windows-setup | cut -d - -f 2)
+        if [ "$GITHUB_REF_NAME" != "main" ]; then VERSION="$VERSION-testing"; fi
+        SIGS_FILE="nzbget-$VERSION.sig.txt"
+
+        echo "Generating $SIGS_FILE ..."
+        echo
+
+        echo "nzbget_signatures({" | tee $SIGS_FILE
+        echo | tee -a $SIGS_FILE
+
+        for FILE in *.exe *.run *.zip *.spk *.qpkg; do
+            [ -f $FILE ] || continue
+
+            MD5=$(openssl dgst -md5 $FILE | cut -d ' ' -f 2)
+            SHA1=$(openssl dgst -sha1 $FILE | cut -d ' ' -f 2)
+            SHA256=$(openssl dgst -rsa-sha256 $FILE | cut -d ' ' -f 2)
+            RSASHA256=$(openssl dgst -rsa-sha256 -sign <(echo "$PRIVATE_KEY") $FILE | hexdump -ve '1/1 "%.2x"')
+
+            echo "\"MD5($FILE)\" : \"$MD5\"," | tee -a $SIGS_FILE
+            echo "\"SHA1($FILE)\" : \"$SHA1\"," | tee -a $SIGS_FILE
+            echo "\"SHA256($FILE)\" : \"$SHA256\"," | tee -a $SIGS_FILE
+            echo "\"RSA-SHA256($FILE)\" : \"$RSASHA256\"," | tee -a $SIGS_FILE
+
+            echo | tee -a $SIGS_FILE
+        done
+
+        echo "\"\" : \"\"});" | tee -a $SIGS_FILE
+
+        cd ..
+        echo
+        echo "Done."
+    
+    - name: Upload build artifacts with signatures
+      uses: actions/upload-artifact@v4
+      with:
+        name: nzbget-installers
+        path: builds/*
+        retention-days: 5
+
+    - name: Delete unneded platform-specific artifacts
+      uses: geekyeggo/delete-artifact@v4
+      with:
+        name: |
+          nzbget-windows-installers
+          nzbget-linux-installers
+          nzbget-osx-installers
+          nzbget-synology-packages
+          nzbget-qnap-packages
+          nzbget-qnap-native-packages
+
+  make-testing-release:
+    runs-on: [self-hosted, linux]
+    needs: [generate-signatures]
+    permissions:
+      contents: write
+    if: github.ref_name == 'develop'
+    steps:
+
+    - name: Delete tag and release
+      uses: dev-drprasad/delete-tag-and-release@v0.2.1
+      with:
+        delete_release: true
+        tag_name: testing
+      env:
+        GITHUB_TOKEN: ${{ github.token }}
+
+    - name: Download build artifacts
+      uses: actions/download-artifact@v4
+
+    - name: Create latest artifacts
+      run: |
+        cp $(find nzbget-installers/ -name *linux.run) nzbget-installers/nzbget-latest-testing-bin-linux.run
+        cp $(find nzbget-installers/ -name *windows-setup.exe) nzbget-installers/nzbget-latest-testing-bin-windows-setup.exe
+
+    - name: Make release
+      uses: ncipollo/release-action@v1
+      with:
+        artifacts: "nzbget-installers/*"
+        generateReleaseNotes: true
+        tag: testing
+        allowUpdates: true
+        prerelease: true
+
+    - name: Update website info
+      run: |
+        bash /build/update-release-info.sh
diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml
@@ -0,0 +1,89 @@
+name: docker build
+
+on:
+  push:
+    branches:
+    - develop
+  workflow_dispatch:
+
+env:
+  REGISTRY_IMAGE: nzbgetcom/nzbget
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    permissions:
+      packages: write
+
+    steps:
+
+    - name: Checkout
+      uses: actions/checkout@v4
+
+    - name: Set up QEMU
+      uses: docker/setup-qemu-action@v3
+
+    - name: Set up Docker Buildx
+      uses: docker/setup-buildx-action@v3
+
+    - name: Login to Docker Hub
+      uses: docker/login-action@v3
+      with:
+        username: ${{ secrets.DOCKERHUB_USERNAME }}
+        password: ${{ secrets.DOCKERHUB_TOKEN }}
+
+    - name: Login to GitHub Container Registry
+      uses: docker/login-action@v3
+      with:
+        registry: ghcr.io
+        username: ${{ github.repository_owner }}
+        password: ${{ secrets.GITHUB_TOKEN }}
+
+    - name: Generate image tags
+      id: gen_tags
+      run: |
+        if [[ "$GITHUB_REF_NAME" == "develop" ]]; then
+          TAG="testing"
+        fi
+        if [[ "$GITHUB_REF_NAME" == "main" ]]; then
+          TAG="latest"
+        fi
+        if [[ $GITHUB_REF == 'refs/tags/'* ]]; then
+          TAG="${GITHUB_REF/refs\/tags\//}"
+        fi
+        if [[ "$TAG" == "" ]]; then
+          TAG="${GITHUB_REF_NAME/\//-}"
+        fi
+        TAGS="${{ env.REGISTRY_IMAGE }}:$TAG,ghcr.io/${{ env.REGISTRY_IMAGE }}:$TAG"
+        echo "tags=$TAGS" >> $GITHUB_OUTPUT
+        echo "version=$TAG" >> $GITHUB_OUTPUT
+    
+    - name: Build and push
+      uses: docker/build-push-action@v5
+      with:
+        context: docker
+        platforms: linux/amd64,linux/arm64,linux/arm/v7
+        provenance: false
+        push: true
+        tags: ${{ steps.gen_tags.outputs.tags }}
+        outputs: "type=image,oci-mediatypes=false,name=${{ env.REGISTRY_IMAGE }}"
+        build-args: |
+          "NZBGET_RELEASE=${{ github.ref_name }}"
+          "MAKE_JOBS=2"
+
+    - name: Update Docker Hub Description
+      uses: peter-evans/dockerhub-description@v3
+      with:
+        username: ${{ secrets.DOCKERHUB_USERNAME }}
+        password: ${{ secrets.DOCKERHUB_TOKEN }}
+        repository: ${{ env.REGISTRY_IMAGE }}
+        readme-filepath: ./docker/README.md
+
+    - name: Cleanup ghcr.io from outdated images
+      uses: miklinux/ghcr-cleanup-action@v1
+      with:
+        token: ${{ secrets.GITHUB_TOKEN }}
+        package-owner: nzbgetcom
+        package-name: nzbget
+        delete-orphans: true
+        dry-run: false
diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml
@@ -1,10 +1,7 @@
 name: linux build 
 
 on:
-  push:
-    branches:
-    - develop
-    - main
+  workflow_call:
   workflow_dispatch:
 
 jobs:
@@ -14,19 +11,39 @@ jobs:
     steps:
 
     - name: Checkout
-      uses: actions/checkout@v3
+      uses: actions/checkout@v4
       with:
         fetch-depth: 0
 
+    - name: Change version for non-release
+      if: github.ref_name != 'main'
+      run: |
+        VERSION=$(cat configure.ac | grep AC_INIT | cut -d , -f 2 | xargs)
+        NEW_VERSION="$VERSION-testing-$(date '+%Y%m%d')"
+        sed -e "s|AC_INIT(nzbget.*|AC_INIT(nzbget, $NEW_VERSION, https://github.com/nzbgetcom/nzbget/issues)|g" -i configure.ac
+        echo NEW_VERSION=$NEW_VERSION >> $GITHUB_ENV
+
     - name: Build
       run: |
         rm -rf /build/nzbget
+        rm -rf /build/output
         cp -r . /build/nzbget
         cd /build
-        docker run -e ALL_ARCHS="i686 x86_64 armhf armel" -v /build:/build nzbget-build /build/scripts/build-nzbget-ci.sh
+        docker run -e ALL_ARCHS="i686 x86_64 aarch64 armhf armel mipseb mipsel ppc6xx ppc500" -v /build:/build nzbget-build /build/scripts/build-nzbget-ci.sh
+
+    - name: Rename build artifacts
+      if: github.ref_name != 'main' && github.ref_name != 'develop'
+      run: |
+        cd /build/output
+        NEW_VERSION_FEATURE="$NEW_VERSION-${GITHUB_REF_NAME/\//-}"
+        for FILE in *.run; do
+          [ -f $FILE ] || continue
+          NEW_FILE=${FILE/$NEW_VERSION/$NEW_VERSION_FEATURE}
+          mv $FILE $NEW_FILE          
+        done
 
     - name: Upload build artifacts
-      uses: actions/upload-artifact@v3
+      uses: actions/upload-artifact@v4
       with:
         name: nzbget-linux-installers
         path: /build/output/*.run

diff --git a/.github/workflows/osx.yml b/.github/workflows/osx.yml
@@ -1,10 +1,7 @@
 name: osx build
 
 on:
-  push:
-    branches:
-    - develop
-    - main
+  workflow_call:
   workflow_dispatch:
 
 jobs:
@@ -14,16 +11,35 @@ jobs:
     steps:
 
     - name: Checkout
-      uses: actions/checkout@v3
+      uses: actions/checkout@v4
       with:
         fetch-depth: 0
+
+    - name: Change version for non-release
+      if: github.ref_name != 'main'
+      run: |
+        VERSION=$(cat configure.ac | grep AC_INIT | cut -d , -f 2 | xargs)
+        NEW_VERSION="$VERSION-testing-$(date '+%Y%m%d')"
+        sed -e "s|AC_INIT(nzbget.*|AC_INIT(nzbget, $NEW_VERSION, https://github.com/nzbgetcom/nzbget/issues)|g" -i '' configure.ac
+        echo NEW_VERSION=$NEW_VERSION >> $GITHUB_ENV
 
     - name: Build
       run: |
-        bash osx/build-nzbget.sh 
+        bash osx/build-nzbget.sh
+
+    - name: Rename build artifacts
+      if: github.ref_name != 'main' && github.ref_name != 'develop'
+      run: |
+        cd osx/build/Release
+        NEW_VERSION_FEATURE="$NEW_VERSION-${GITHUB_REF_NAME/\//-}"
+        for FILE in *.zip; do
+          [ -f $FILE ] || continue
+          NEW_FILE=${FILE/$NEW_VERSION/$NEW_VERSION_FEATURE}
+          mv $FILE $NEW_FILE
+        done
 
     - name: Upload build artifacts
-      uses: actions/upload-artifact@v3
+      uses: actions/upload-artifact@v4
       with:
         name: nzbget-osx-installers
         path: osx/build/Release/*.zip