Create Google CDN Uploads App

docs/decisions/0017-reimplement-asset-processing.rst

@@ -131,17 +131,17 @@ The three top-level edx-platform asset processing actions are *build*, *collect*
      - ``scripts/build-assets.sh``
 
        A Bash script that contains all build stages, with subcommands available for running each stage separately. Its command-line interface inspired by Tutor's ``openedx-assets`` script. The script will be runnable on any POSIX system, including macOS and Ubuntu and it will linted for common shell scripting mistakes using `shellcheck <https://www.shellcheck.net>`_.
-     
+
    * - + **Build stage 1: Copy npm-installed assets** from node_modules to other folders in edx-platform. They are used by certain especially-old legacy LMS & CMS frontends that are not set up to work with npm directly.
 
      - ``paver update_assets --skip-collect``
 
        Implemented in Python within update_assets. There is no standalone command for it.
 
-     - ``scripts/build-assets.sh npm``
+     - ``npm install``
+
+       An NPM post-install hook will automatically call scripts/copy-node-modules.sh, a pure Bash reimplementation of the node_modules asset copying, whenever ``npm install`` is invoked.
 
-       Pure Bash reimplementation. See *Rejected Alternatives* for a note about this.
-
    * - + **Build stage 2: Copy XModule fragments** from the xmodule source tree over to input directories for Webpack and SCSS compilation. This is required for a hard-coded list of old XModule-style XBlocks. This is not required for new pure XBlocks, which include (or pip-install) their assets into edx-platform as ready-to-serve JS/CSS/etc fragments.
 
      - ``paver process_xmodule_assets``, or
@@ -156,7 +156,7 @@ The three top-level edx-platform asset processing actions are *build*, *collect*
 
        + `Reimplement this step in Bash <https://github.com/openedx/edx-platform/issues/31611>`_.
        + `Remove the need for this step entirely <https://github.com/openedx/edx-platform/issues/31624>`_.
-   
+
    * - + **Build stage 3: Run Webpack** in order to to shim, minify, otherwise process, and bundle JS modules. This requires a call to the npm-installed ``webpack`` binary.
 
      - ``paver webpack``
@@ -168,7 +168,7 @@ The three top-level edx-platform asset processing actions are *build*, *collect*
        Bash wrapper around a call to webpack. The script will accept parameters rather than looking up Django settings itself.
 
        The print_setting command will still be available for distributions to use to extract ``STATIC_ROOT`` from Django settings, but it will only need to be run once. As described in **Build Configuration** below, unnecessary Django settings will be removed. Some distributions may not even need to look up ``STATIC_ROOT``; Tutor, for example, will probably render ``STATIC_ROOT`` directly into the environment variable ``OPENEDX_BUILD_ASSETS_OPTS`` variable, described in the **Build Configuration**.
-   
+
    * - + **Build stage 4: Compile default SCSS** into CSS for legacy LMS/CMS frontends.
 
      - ``paver compile_sass``
@@ -180,7 +180,7 @@ The three top-level edx-platform asset processing actions are *build*, *collect*
      - ``scripts/build-assets.sh css``
 
        Bash reimplementation, calling ``node-sass`` and ``rtlcss``.
-   
+
        The initial implementation of build-assets.sh may use ``sassc``, a CLI provided by libsass, instead of node-sass. Then, ``sassc`` can be replaced by ``node-sass`` as part of a subsequent `edx-platform frontend framework upgrade effort <https://github.com/openedx/edx-platform/issues/31616>`_.
 
    * - + **Build stage 5: Compile themes' SCSS** into CSS for legacy LMS/CMS frontends. The default SCSS is used as a base, and theme-provided SCSS files are used as overrides. Themes are searched for from some number of operator-specified theme directories.
@@ -198,7 +198,7 @@ The three top-level edx-platform asset processing actions are *build*, *collect*
        The management command will remain available, but it will need to be updated to point at the Bash script, which will replace the paver task (see build stage 4 for details).
 
        The overall asset *build* action will use the Bash script; this means that list of theme directories will need to be provided as arguments, but it ensures that the build can remain Python-free.
-   
+
    * - **Collect** the built static assets from edx-platform to another location (the ``STATIC_ROOT``) so that they can be efficiently served *without* Django's webserver. This step, by nature, requires Python and Django in order to find and organize the assets, which may come from edx-platform itself or from its many installed Python and NPM packages. This is only needed for **production** environments, where it is usually desirable to serve assets with something efficient like NGINX.
 
      - ``paver update_assets``
@@ -210,7 +210,7 @@ The three top-level edx-platform asset processing actions are *build*, *collect*
      - ``./manage.py lms collectstatic --noinput && ./manage.py cms collectstatic --noinput``
 
        The standard Django interface will be used without a wrapper. The ignore patterns will be added to edx-platform's `staticfiles app configuration <https://docs.djangoproject.com/en/4.1/ref/contrib/staticfiles/#customizing-the-ignored-pattern-list>`_ so that they do not need to be supplied as part of the command.
-   
+
    * - **Watch** static assets for changes in the background. When a change occurs, rebuild them automatically, so that the Django webserver picks up the changes. This is only necessary in **development** environments. A few different sets of assets may be watched: XModule fragments, Webpack assets, default SCSS, and theme SCSS.
 
      - ``paver watch_assets``
@@ -221,7 +221,9 @@ The three top-level edx-platform asset processing actions are *build*, *collect*
 
        (where ``<stage>`` is optionally one of the build stages described above. If provided, only that stage's assets will be watched.)
 
-       Bash wrappers around invocation(s) of `watchman <https://facebook.github.io/watchman/>`_, a popular file-watching library maintained by Meta. Watchman is already installed into edx-platform (and other services) via the pywatchman pip wrapper package.
+       Bash wrappers around invocations of the `watchdog library <https://pypi.org/project/watchdog/>`_ for themable/themed assets, and `webpack --watch <https://webpack.js.org/configuration/watch/>`_ for Webpack-managed assets. Both of these tools are available via dependencies that are already installed into edx-platform.
+
+       We considered using `watchman <https://facebook.github.io/watchman/>`_, a popular file-watching library maintained by Meta, but found that the Python release of the library is poorly maintained (latest release 2017) and the documentation is difficult to follow. `Django uses pywatchman but is planning to migrate off of it <https://code.djangoproject.com/ticket/34479>`_ and onto `watchfiles <https://pypi.org/project/watchfiles/>`_. We considered watchfiles, but decided against adding another developer dependency to edx-platform. Future developers could consider migrating to watchfiles if it seemed worthwile.
 
        Note: This adds a Python dependency to build-assets.sh. However, we could be clear that watchman is an *optional* dependency of build-assets.sh which enables the optional ``--watch`` feature. This would keep the *build* action Python-free. Alternatively, watchman is also available Python-free via apt and homebrew.
 
@@ -300,7 +302,7 @@ Either way, the migration path is straightforward:
    * - ``openedx-assets build``
      - ``scripts/build-assets.sh``
    * - ``openedx-assets npm``
-     - ``scripts/build-assets.sh npm``
+     - ``scripts/copy-node-modules.sh  # (automatically invoked by 'npm install'!)
    * - ``openedx-assets xmodule``
      - ``scripts/build-assets.sh xmodule``
    * - ``openedx-assets common``
@@ -328,13 +330,6 @@ OpenCraft has also performed a discovery on a `modernized system for static asse
 Rejected Alternatives
 *********************
 
-Copy node_modules via npm post-install
-======================================
-
-It was noted that `npm supports lifecycle scripts <https://docs.npmjs.com/cli/v6/using-npm/scripts#pre--post-scripts>`_ in package.json, including ``postinstall``. We could use a post-install script to copy assets out of node_modules; this would occurr automatically after ``npm install``. Arguably, this would be more idiomatic than this ADR's proposal of ``scripts/build-assets.sh npm``.
-
-For now, we decided against this. While it seems like a good potential future improvement, we are currently unsure how it would interact with `moving node_modules out of edx-platform in Tutor <https://github.com/openedx/wg-developer-experience/issues/150>`_, which is a motivation behind this ADR. For example, if node_modules could be located anywhere on the image, then we are not sure how the post-install script could know its target directory without us hard-coding Tutor's directory structure into the script.
-
 Live with the problem
 ======================
 

openedx/features/google_cdn_uploads/__init__.py

@@ -0,0 +1 @@
+default_app_config = "openedx.features.google_cdn_uploads.apps:GoogleCDNUploadsConfig"

openedx/features/google_cdn_uploads/admin.py

@@ -0,0 +1,17 @@
+"""
+Admin registration for Messenger.
+"""
+from django.contrib import admin
+
+from openedx.features.google_cdn_uploads.models import GoogleCDNUpload
+
+
+class GoogleCDNUploadAdmin(admin.ModelAdmin):
+    """
+    Admin config clearesult credit providers.
+    """
+    list_display  = ("file_name", "course_id",)
+    search_fields = ("file_name", "course_id",)
+
+
+admin.site.register(GoogleCDNUpload, GoogleCDNUploadAdmin)

openedx/features/google_cdn_uploads/apps.py

@@ -0,0 +1,24 @@
+"""
+Meta Translations App Config
+"""
+from django.apps import AppConfig
+from edx_django_utils.plugins import PluginURLs, PluginSettings
+from openedx.core.djangoapps.plugins.constants import ProjectType, SettingsType
+
+class GoogleCDNUploadsConfig(AppConfig):
+    name = 'openedx.features.google_cdn_uploads'
+    plugin_app = {
+        PluginURLs.CONFIG: {
+            ProjectType.CMS: {
+                    PluginURLs.NAMESPACE: 'google_cdn_uploads',
+                    PluginURLs.REGEX: '^google_cdn_uploads/',
+                    PluginURLs.RELATIVE_PATH: 'urls',
+                },
+        },
+        PluginSettings.CONFIG: {
+            ProjectType.CMS: {
+                SettingsType.COMMON: {PluginSettings.RELATIVE_PATH: 'settings.common'},
+            },
+        }
+    }
+

openedx/features/google_cdn_uploads/models.py

@@ -0,0 +1,17 @@
+"""
+Meta Translations Models
+"""
+from django.db import models
+from opaque_keys.edx.django.models import CourseKeyField
+
+
+class GoogleCDNUpload(models.Model):
+    """
+    Store course id and file name
+    """
+    course_id = CourseKeyField(max_length=255, db_index=True)
+    file_name = models.CharField(max_length=255)
+
+    class Meta:
+        app_label = 'google_cdn_uploads'
+        verbose_name = "Google CDN Uploads"

openedx/features/google_cdn_uploads/settings/common.py

@@ -0,0 +1,17 @@
+"""
+Common settings for Messenger
+"""
+
+
+def plugin_settings(settings):
+    """
+    Common settings for uploading mp4
+    """
+    settings.MAKO_TEMPLATE_DIRS_BASE.append(
+      settings.OPENEDX_ROOT / 'features' / 'google_cdn_uploads' / 'templates',
+    )
+
+    settings.STATICFILES_DIRS.append (
+      settings.OPENEDX_ROOT / 'features' / 'google_cdn_uploads' / 'static',
+    )
+    print('CDN App - Setting Updated')

openedx/features/google_cdn_uploads/static/google_cdn_uploads/js/GoogleCDNUpload.js

@@ -0,0 +1,11 @@
+import React from 'react';
+import ReactDOM from 'react-dom';
+import UploadContext from "./UploadContext";
+
+export default class GoogleCDNUpload {
+  constructor() {
+    ReactDOM.render(<UploadContext context={context} />, document.getElementById("root"));
+  }
+}
+
+export { GoogleCDNUpload }

openedx/features/google_cdn_uploads/static/google_cdn_uploads/js/UploadContext.js

@@ -0,0 +1,11 @@
+import React from 'react';
+
+function UploadContext({ context }) {
+  return (
+    <div className="translations">
+      <p>kjasnxkajsnkxajns</p>
+    </div>
+  )
+}
+
+export default UploadContext;

openedx/features/google_cdn_uploads/templates/uploads.html

@@ -0,0 +1,27 @@
+<%inherit file="base.html" />
+<%namespace name='static' file='static_content.html'/>
+<%!
+  from django.urls import reverse
+  from django.utils.translation import ugettext as _
+  from openedx.core.djangolib.js_utils import js_escaped_string, dump_js_escaped_json
+%>
+
+<%def name="online_help_token()"><% return "home" %></%def>
+<%block name="title">${_("{studio_name} Home").format(studio_name=settings.STUDIO_SHORT_NAME)}</%block>
+<%block name="bodyclass">is-signedin index view-dashboard</%block>
+
+<%block name="header_extras">
+    <link rel="stylesheet" type="text/css" href="${static.url('css/studio-main-v1.css')}" />
+</%block>
+
+<%block name="content">
+    <div id="root" class="container">
+        hello
+    </div>
+
+    <%static:webpack entry="GoogleCDNUpload">
+        var context = {}
+
+        new GoogleCDNUpload(context);
+    </%static:webpack>
+</%block>

openedx/features/google_cdn_uploads/urls.py

@@ -0,0 +1,21 @@
+"""
+Urls for Meta Translations
+"""
+from django.conf.urls import include, url
+
+from  openedx.features.google_cdn_uploads.views import render_google_cdn_uploads_home
+
+app_name = 'google_cdn_uploads'
+
+urlpatterns = [
+    # url(
+    #     r'^course_blocks_mapping/$',
+    #     course_blocks_mapping,
+    #     name='course_blocks_mapping'
+    # ),
+    url(
+        r'^$',
+        render_google_cdn_uploads_home,
+        name='google_cdn_uploads_home'
+    ),
+]

openedx/features/google_cdn_uploads/views.py

@@ -0,0 +1,23 @@
+"""
+Views for uploading mp4
+"""
+import json
+from logging import getLogger
+from django.contrib.auth.decorators import login_required
+from django.views.decorators.http import require_http_methods
+from django.http import JsonResponse
+from xmodule.modulestore.django import modulestore
+from opaque_keys.edx.keys import CourseKey
+from django.conf import settings
+from common.djangoapps.edxmako.shortcuts import render_to_response
+
+from lms.djangoapps.courseware.courses import get_course_by_id
+log = getLogger(__name__)
+
+
+@login_required
+def render_google_cdn_uploads_home(request):
+    return render_to_response('uploads.html', {
+        'uses_bootstrap': True,
+        'login_user_username': request.user.username,
+    })

package.json

@@ -2,6 +2,18 @@
   "name": "edx",
   "version": "0.1.0",
   "repository": "https://github.com/openedx/edx-platform",
+  "scripts": {
+    "postinstall": "scripts/copy-node-modules.sh",
+    "build": "echo 'WARNING: `npm run build` in edx-platform is experimental. Use at your own risk.' && npm run webpack && npm run compile-sass",
+    "build-dev": "echo 'WARNING: `npm run build-dev` in edx-platform is experimental. Use at your own risk.' && npm run webpack-dev && npm run compile-sass-dev",
+    "webpack": "NODE_ENV=${NODE_ENV:-production} \"$(npm bin)/webpack\" --config=${WEBPACK_CONFIG_PATH:-webpack.prod.config.js}",
+    "webpack-dev": "NODE_ENV=development \"$(npm bin)/webpack\" --config=webpack.dev.config.js",
+    "compile-sass": "scripts/compile_sass.py --env=${NODE_ENV:-production}",
+    "compile-sass-dev": "scripts/compile_sass.py --env=development",
+    "watch": "echo 'WARNING: `npm run watch` in edx-platform is experimental. Use at your own risk.' && { npm run watch-webpack& npm run watch-sass& } && sleep infinity",
+    "watch-webpack": "npm run webpack-dev -- --watch",
+    "watch-sass": "scripts/watch_sass.sh"
+  },
   "dependencies": {
     "@babel/core": "7.19.0",
     "@babel/plugin-proposal-object-rest-spread": "^7.18.9",