Add live feedback storage and statistics #7497

syoopie · 2024-08-19T09:45:32Z

Add support for saving live feedback and displaying it in the assessment statistics page.

Workflow for saving:

GetHelp used, FE sends request to BE.
New entry in course_assessment_live_feedbacks is created and current code states (course_assessment_live_feedback_code) are saved
BE requests for live feedback
1. If feedback is returned:
  1. Save feedback (course_assessment_live_feedback_comments)
  2. Send feedback to FE
2. If token is returned:
  1. Send token to FE
  2. FE polls for feedback
  3. FE sends feedback to BE to be saved (course_assessment_live_feedback_comments)

Schema Changes: Added 3 new tables

feedback_id under the course_assessments_live_feedbacks table corresponds to the transaction_id provided by codaveri. This is to ensure traceability.

The schema is separated as such so that each feedback request can accomadate multiple different code files, and each code file can have multiple comments.

New GetHelp statistics page:

Table is sorted by descending total
Individual questions are colour coded

Other Notes:

This PR also standardises the terminology surrounding the live feedback function.

From now on we will only use 2 terms, "Get Help" and "Live Feedback"
"Get Help" will be an active, user facing term, which will be used for buttons and their corresponding translations.
"Live Feedback" will be the technical term, used for describing the feature to course managers etc., and will also be the term used in the codebase when referring to this feature

bivanalhar

Hi @syoopie good job on your PR! It's quite well made..
I have left some comments inside this PR, please attend to them and apply the changes accordingly. If you have any further inquiries, please reply to my comments and if you don't understand how to apply those changes, please don't hesitate to reach out to me

Other than that, here are some advise for your future PRs:

please properly name your commit. We have the convention written in our Commit Guideline. Not sure if you have read this before, but please learn about this
avoid creating a new commit that are actually merely fixing the commit you just made previously that are still within the same PR. If you have committed several more commits then realising your mistake, instead do git rebase and then edit the respective commit.. Please take a look at the following git rebase guideline to know how to use git rebase properly

Please let me know when you're finished polishing your code, and then request for my re-review for me to look more into your PR

app/models/course/assessment/question.rb

db/migrate/20240808083848_add_live_feedback_code_and_comments.rb

db/schema.rb

app/views/course/statistics/assessments/main_statistics.json.jbuilder

app/models/course/assessment.rb

app/controllers/concerns/course/statistics/live_feedback_concern.rb

db/migrate/20240808083848_add_live_feedback_code_and_comments.rb

app/models/course/assessment/assessment_ability.rb

...les/course/assessment/pages/AssessmentStatistics/LiveFeedbackHistory/LiveFeedbackDetails.tsx

...course/assessment/pages/AssessmentStatistics/LiveFeedbackHistory/LiveFeedbackHistoryPage.tsx

client/app/types/course/statistics/assessmentStatistics.ts

app/services/course/assessment/answer/programming_codaveri_async_feedback_service.rb

client/app/bundles/course/assessment/pages/AssessmentStatistics/LiveHelpStatisticsTable.tsx

bivanalhar · 2024-09-10T15:38:19Z

app/controllers/course/assessment/submission/submissions_controller.rb

+      @answer.actable.files
+    )
+
+    if response_status == 200


@syoopie please attend to this review. Btw, just now I tried to comment out that line save_live_feedback and the API in generating live feedback still runs as expected (when you try to generate live feedback, that generated feedback will be saved accordingly to our DB and the count is increased by 1)

Therefore, please investigate once more if the line 116 is needed here.

bivanalhar · 2024-09-10T15:41:08Z

client/app/bundles/course/assessment/submission/actions/answers/index.js

@@ -318,6 +321,11 @@ export function fetchLiveFeedback({
      .fetchLiveFeedback(feedbackUrl, feedbackToken)
      .then((response) => {
        if (response.status === 200) {
+          CourseAPI.assessment.submissions.saveLiveFeedback(


I mean this one also calls that controller function save_live_feedback, so with having that save_live_feedback on line 116 of course/assessment/submission/submissions_controller.rb, technically this function is called twice. I don't see the reason why this should happen

On the other hand, deactivating this API call while keeping that line 116 intact makes the Get Help count to be stagnant (not increased) when Get Help is clicked. Please investigate further and explain why this occurs

Backend only does the saving when the feedback is immediately returned from codaveri.
The only case when frontend does saving is when it needs to poll codaveri. In this scenario backend does not save the feedback (response status will be 201)

client/app/bundles/course/assessment/pages/AssessmentStatistics/AssessmentStatisticsPage.tsx

client/app/bundles/course/assessment/pages/AssessmentStatistics/utils.js

bivanalhar

Okay nice! @cysjonathan I think from me, the review's done for this PR. Please review it once more and once you approve this, should be good to merge

…s enabled

New Tables: 1. course_assessment_live_feedbacks 2. course_assessment_live_feedback_code 3. course_assessment_live_feedback_comments Related models have also been updated. These include: users course_assessments course_assessment_questions

add new route for getting live feedback stats add new live_feedback_concern refactored some code in assessments_controller as it was used for live feedback stats as well add new json return format for live feedback stats

refactored for a wider range of usability fix erroneous comments add new function to get class for live feedback cells previously cells were coloured linearly based on a max number new colouring only applies this gradient to the lower 75% of the data upper 25% is all coloured the most intense shade of red

add new tab to assessment statistics if live help is enabled for assessment add new types for live feedback statistics add call to new route for getting live feedback statistics refactor functionality for getting jointGroupsName refactor functionality for translating status both refactors have their commented out typescript equivalent left inside in file

this commit used to be to fix a breaking change from codaveri regarding categories, but the change has been made by another commit that is already merged the change has been removed from this commit during rebase and only other random changes are left remove some random commented code in codaveri-async-feedback-service

add frontend logic for returning live feedback once retrieved add backend logic and routes for saving live feedback

individual live feedback histories need this information to be retrieved

add dialog to show live feedback history on the live feedback statistics page add routes and backend for retrieving live feedback history

Change assessments factory to give unique question weights when creating questions This is to assist in live feedback tests as weights are needed to order the questions

abstract out the translations of some assessment stsatistics files add translations and localizations for the associated translations

- conform to new standard, all user facing "verbs" are "Get Help" - "Get Help" will describe the act of getting live feedback - all other usage of the terms "Live Help" and "Get Help" will instead be "Live Feedback"

add custom slider with more visible marks and larger height

- sql queries drop from 99 -> ~30

syoopie changed the title ~~yupei/add live feedback storage and statistics~~ Add live feedback storage and statistics Aug 19, 2024

syoopie force-pushed the yupei/add-live-feedback-storage-and-statistics branch 2 times, most recently from 889de1f to 389bdac Compare August 20, 2024 02:47

bivanalhar requested review from bivanalhar and removed request for bivanalhar August 20, 2024 07:14

syoopie force-pushed the yupei/add-live-feedback-storage-and-statistics branch from 389bdac to 063e585 Compare August 20, 2024 07:46

syoopie marked this pull request as draft August 27, 2024 10:29

syoopie mentioned this pull request Aug 27, 2024

yupei/add live feedback history #7511

Merged

syoopie force-pushed the yupei/add-live-feedback-storage-and-statistics branch 4 times, most recently from 302abf2 to fb7f80f Compare August 29, 2024 05:33

syoopie marked this pull request as ready for review August 29, 2024 05:36

syoopie requested a review from bivanalhar August 29, 2024 05:36

bivanalhar reviewed Sep 3, 2024

View reviewed changes

syoopie force-pushed the yupei/add-live-feedback-storage-and-statistics branch 5 times, most recently from 0f76c0b to d5e7640 Compare September 3, 2024 09:46

bivanalhar reviewed Sep 3, 2024

View reviewed changes

syoopie force-pushed the yupei/add-live-feedback-storage-and-statistics branch 7 times, most recently from bd8c12a to 85bdef9 Compare September 3, 2024 14:25

bivanalhar reviewed Sep 3, 2024

View reviewed changes

syoopie force-pushed the yupei/add-live-feedback-storage-and-statistics branch from 85bdef9 to e1cb9b4 Compare September 4, 2024 03:01

bivanalhar force-pushed the yupei/add-live-feedback-storage-and-statistics branch from 52a2886 to 0776582 Compare September 10, 2024 15:20

bivanalhar requested changes Sep 10, 2024

View reviewed changes

syoopie force-pushed the yupei/add-live-feedback-storage-and-statistics branch from 0776582 to 39254a6 Compare September 11, 2024 06:12

bivanalhar self-requested a review September 11, 2024 06:27

bivanalhar approved these changes Sep 11, 2024

View reviewed changes

bivanalhar requested a review from cysjonathan September 11, 2024 06:28

syoopie force-pushed the yupei/add-live-feedback-storage-and-statistics branch 3 times, most recently from 71e017b to 896d547 Compare September 16, 2024 11:18

syoopie added 15 commits September 17, 2024 12:28

refactor(main-statistics): add boolean to indicate if live feedback i…

019dc23

…s enabled

feat(live-feedback-stats): add backend

7ede67f

add new route for getting live feedback stats add new live_feedback_concern refactored some code in assessments_controller as it was used for live feedback stats as well add new json return format for live feedback stats

feat(live-feedback): add live feedback saving

940e65f

add frontend logic for returning live feedback once retrieved add backend logic and routes for saving live feedback

refactor(submission-controller): disable file length restrictions

00d3ad9

refactor(EditorField): add optional cursorStart variable

e303b8e

refactor(live-feedback-stats): add retrieval of question ids

8501a38

individual live feedback histories need this information to be retrieved

feat(live-feedback-history): add FE and BE

ba5e6f7

add dialog to show live feedback history on the live feedback statistics page add routes and backend for retrieving live feedback history

test(live-feedback-history): add tests for live feedback history

0509bd2

refactor(assessments-factory)

a6e05d6

Change assessments factory to give unique question weights when creating questions This is to assist in live feedback tests as weights are needed to order the questions

test(live-feedback-statistics) add tests for live feedback statistics

4be98c7

refactor(assessment-statistics): abstract translations

6a9f0ea

abstract out the translations of some assessment stsatistics files add translations and localizations for the associated translations

syoopie force-pushed the yupei/add-live-feedback-storage-and-statistics branch from 896d547 to 43e0ae0 Compare September 17, 2024 04:28

syoopie added 4 commits September 17, 2024 14:41

style(live-feedback): change translations

082bd40

- conform to new standard, all user facing "verbs" are "Get Help" - "Get Help" will describe the act of getting live feedback - all other usage of the terms "Live Help" and "Get Help" will instead be "Live Feedback"

style(live-feedback-history): improve slider styling

01fee01

add custom slider with more visible marks and larger height

style(assessment-statistics): use new custom slider

01ff5f3

perf(live-feedback-stats): reduce N+1

c85a081

- sql queries drop from 99 -> ~30

syoopie force-pushed the yupei/add-live-feedback-storage-and-statistics branch from 43e0ae0 to c85a081 Compare September 17, 2024 06:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add live feedback storage and statistics #7497

Add live feedback storage and statistics #7497

syoopie commented Aug 19, 2024 •

edited

Loading

bivanalhar left a comment •

edited

Loading

bivanalhar Sep 10, 2024

bivanalhar Sep 10, 2024

syoopie Sep 11, 2024

bivanalhar left a comment

Add live feedback storage and statistics #7497

Are you sure you want to change the base?

Add live feedback storage and statistics #7497

Conversation

syoopie commented Aug 19, 2024 • edited Loading

Workflow for saving:

Schema Changes: Added 3 new tables

New GetHelp statistics page:

Other Notes:

bivanalhar left a comment • edited Loading

Choose a reason for hiding this comment

bivanalhar Sep 10, 2024

Choose a reason for hiding this comment

bivanalhar Sep 10, 2024

Choose a reason for hiding this comment

syoopie Sep 11, 2024

Choose a reason for hiding this comment

bivanalhar left a comment

Choose a reason for hiding this comment

syoopie commented Aug 19, 2024 •

edited

Loading

bivanalhar left a comment •

edited

Loading