Add eligible segment filter for bandits #13395

moritzhaller · 2021-01-06T16:45:53Z

No description provided.

kjozwiak · 2021-01-26T00:28:28Z

Labelling as Android as per confirmation from @tmancey & @moritzhaller from today's meeting that epsilon greedy bandit classifier will also be on Android. Also, #12595 was labelled with OS/Android as well.

stephendonner · 2021-02-04T00:12:42Z

Verified using the test plan in brave/brave-core#7533 with additional contextual details, from @moritzhaller and @tmancey (and thx to @btlechowski for pro-tips, too!)

Brave	1.20.97 Chromium: 88.0.4324.96 (Official Build) dev (x86_64)
Revision	68dba2d8a0b149a1d3afac56fa74648032bcf46b-refs/branch-heads/4324@{#1784}

Case 1: Default models enabled (Text Classification and Purchase Intent)

Used --args --enable-logging=stderr --vmodule="*/bat-native-ledger/*"=6,"*/brave_rewards/*"=6,"*/bat-native-ads/*"=6,"*/bat-native-confirmations/*"=6,"*/brave_ads/*"=6,"*/brave_user_model/*"=6 --brave-ads-staging --brave-ads-debug --rewards=staging=true,reconcile-interval=5

Verify that Text Classification and Purchase Intent models are enabled and Epsilon Greedy Bandit is disabled

[14716:775:0129/091938.057839:VERBOSE1:features.cc(61)] Epsilon greedy bandit feature is disabled
[14716:775:0129/091938.057884:VERBOSE1:features.cc(63)] Purchase intent feature is enabled

Trigger ad notification and verify that ads are served as usual via text classification

[14716:775:0129/092959.983897:VERBOSE1:text_classification_processor.cc(62)] Classified text with the top segment as gaming-gaming
[14716:775:0129/093025.448373:VERBOSE1:ad_notification_serving.cc(101)] Ad notification delivered:
  uuid: f41fdce0-62b0-42dc-99b6-77fe84db5083
  creativeInstanceId: ed6c467a-8643-42eb-a7cd-32e878fa3cd4
  creativeSetId: a9f0b49c-3298-4c1b-a8b5-76a71e8d7332
  campaignId: 0e70c893-b0c0-4dab-860d-7e65446f0f0f
  segment: untargeted
  title: This is a test
  body: This is a test
  targetUrl: https://www.brave.com
[14716:775:0129/093025.448421:VERBOSE3:ad_notification_event_viewed.cc(24)] Viewed ad notification with uuid f41fdce0-62b0-42dc-99b6-77fe84db5083 and creative instance id ed6c467a-8643-42eb-a7cd-32e878fa3cd4
[14716:775:0129/093025.448808:VERBOSE1:confirmations.cc(90)] Confirming view ad for creative instance id ed6c467a-8643-42eb-a7cd-32e878fa3cd4

Process purchase intent segment and verify keywords are extracted

[14716:775:0129/091938.122877:VERBOSE1:purchase_intent_resource.cc(58)] Successfully loaded kkjipiepeooghlclkedllogndmohhnhi purchase intent resource
[14716:775:0129/091938.124779:VERBOSE1:purchase_intent_resource.cc(235)] Parsed purchase intent user model version 1
[14716:775:0129/091938.125023:VERBOSE1:purchase_intent_resource.cc(68)] Successfully initialized kkjipiepeooghlclkedllogndmohhnhi purchase intent resource

Verify that bandit is processing (despite the model being disabled, that is by design)

[14716:775:0129/093027.940733:VERBOSE1:epsilon_greedy_bandit_processor.cc(118)] Epsilon greedy bandit processed BraveAdsAdNotificationEventType::kClicked
[14716:775:0129/093027.947347:VERBOSE1:ad_notification_event_clicked.cc(36)] Successfully logged ad notification clicked event

Verify that the model is not suggesting segments to the ad serve pipeline

Nothing shows up under Exploring... or Exploiting...

Case 2: Variant models enabled (Epsilon Greedy Bandit and Purchase Intent: --disable-features="TextClassification" --enable-features="EpsilonGreedyBandit")

Used --args --enable-logging=stderr --vmodule="*/bat-native-ledger/*"=6,"*/brave_rewards/*"=6,"*/bat-native-ads/*"=6,"*/bat-native-confirmations/*"=6,"*/brave_ads/*"=6,"*/brave_user_model/*"=6 --brave-ads-debug --disable-features="TextClassification" --enable-features="EpsilonGreedyBandit"

Verify that Epsilon Greedy Bandit and Purchase Intent models are enabled and Text Classification is disabled

[15207:775:0129/103356.166699:VERBOSE1:features.cc(59)] Text classification feature is disabled
[15207:775:0129/103356.166742:VERBOSE1:features.cc(61)] Epsilon greedy bandit feature is enabled
[15207:775:0129/103356.166784:VERBOSE1:features.cc(63)] Purchase intent feature is enabled

Process purchase intent segment and verify keywords are extracted

[15207:775:0129/104513.181256:VERBOSE1:purchase_intent_processor.cc(101)] Extracted purchase intent signal from visited URL

Verify that bandit is processing

[15207:775:0129/103356.123110:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for architecture segment
(The above repeats for each segment)

Verify that the model is suggesting segments to the ad serve pipeline

[15207:775:0129/104244.918132:VERBOSE2:epsilon_greedy_bandit_model.cc(165)] Exploiting epsilon greedy bandit segments:
[15207:775:0129/104244.920762:VERBOSE1:ad_notification_serving.cc(160)] No segments to serve targeted ads
[15207:775:0129/104244.920807:VERBOSE1:ad_notification_serving.cc(226)] Serve untargeted ad

Case 3: Restart after one round of training and make sure state wasn't deleted in user Preferences JSON key epsilon_greedy_bandit_arms

In addition to the details below, I verified that the pulls and values data persisted between sessions/restarts of the browser, and were accurately reflected via brave://rewards 7-day History panel.

Used --args --enable-logging=stderr --vmodule="*/bat-native-ledger/*"=6,"*/brave_rewards/*"=6,"*/bat-native-ads/*"=6,"*/bat-native-confirmations/*"=6,"*/brave_ads/*"=6,"*/brave_user_model/*"=6 --brave-ads-debug --disable-features="TextClassification" --enable-features="EpsilonGreedyBandit"

Here, I'm covering those ads from segments pulled from the catalog (and shown), which I either clicked or dismissed.
Represented as a percentage (%), clicking will retain or increase (if multiple pulls and views) the value, and conversely, dismissing or timing out will decrease the value.

So, with 2 pulls for a particular segment, but only 1 click, value 0.5 = 50% click-through rate. With only 1 pull and 1 click, value = 1.0 (100%). 2 pulls without clicks will set value to 0.0 (0%).

Before training, each segment's (ad) pulls is set to 0, and its value is set to 1.0 (100%).

key for the below: segment category, ad name, clicked or dismissed (pulls, value, as a % of click-throughs)

arts & entertainment, Splinterlands, dismissed (1 pull, value 0.0 = 0%)
business, Money Metals Exchange, dismissed (1 pull, value 0.0 = 0%)
cell phones, Stay Connected w/SIMPLE, clicked (1 pull, value 1.0 = 100%)
education, BAT = Chances to Win, clicked (1 pull, value 1.0 = 100%)
family & parenting, Lustre for Brave, clicked (1 pull, value 1.0 = 100%)
general (unlisted in bandit training; from new tab page), 6 Gala Games impressions (new tab page), clicked 1 (not represented here in JSON/pulls)
hobbies & interests (1/2), Gemini, dismissed (1st of 2 total pulls, total value 0.5 = 50%)
hobbies & interests (2/2), Gala Games, clicked (2nd of 2 total pulls, total value 0.5 = 50%)
personal finance (1/2), BAT: First Private Ad Network, clicked (1st of 2 total pulls, total value 1.0 = 100%)
personal finance (2/2), Lending Tree, clicked (2nd of 2 total pulls, value 1.0 = 100%)
travel, Coinbase, dismissed (1 pull, value 0.0 = 0%)

References:

a) before and after training comparison, in Preferences.json, found in ~/Library/Application Support/BraveSoftware/Brave-Browser-Dev/Default/Preferences/

b) relevant brave://rewards/ ad history

Verification passed on

Brave	1.20.97 Chromium: 88.0.4324.96 (Official Build) dev (64-bit)
Revision	68dba2d8a0b149a1d3afac56fa74648032bcf46b-refs/branch-heads/4324@{#1784}
OS	Ubuntu 18.04 LTS

Verified using the test plan in brave/brave-core#7533

Default models enabled (Text Classification and Purchase Intent)

Verify that Text Classification and Purchase Intent models are enabled and Epsilon Greedy Bandit is disabled

[2792:2792:0208/234131.018377:VERBOSE1:features.cc(59)] Text classification feature is enabled
[2792:2792:0208/234131.018400:VERBOSE1:features.cc(61)] Epsilon greedy bandit feature is disabled
[2792:2792:0208/234131.018435:VERBOSE1:features.cc(63)] Purchase intent feature is enabled

Trigger ad notification and verify that ads are served as usual via text classification

[2792:2792:0208/234320.947896:VERBOSE1:text_classification_processor.cc(62)] Classified text with the top segment as technology & computing-software
[4134:4134:0208/235705.246825:VERBOSE1:ad_notification_serving.cc(165)] Serve ad for segments:
[4134:4134:0208/235705.246953:VERBOSE1:ad_notification_serving.cc(167)]   technology & computing-software
[4134:4134:0208/235705.247044:VERBOSE1:ad_notification_serving.cc(167)]   personal finance-personal finance
[4134:4134:0208/235705.247144:VERBOSE1:ad_notification_serving.cc(167)]   home-appliances
[4134:4134:0208/235705.248500:VERBOSE2:ad_notification_serving.cc(273)] 1 eligible ads before pacing
[4134:4134:0208/235705.248621:VERBOSE2:eligible_ads_priority_filter.cc(81)] 1 eligible ads with a priority of 1
[4134:4134:0208/235705.248736:VERBOSE2:ad_notification_serving.cc(278)] 1 eligible ads after pacing
[4134:4134:0208/235705.248838:VERBOSE1:ad_notification_serving.cc(264)] Found 1 eligible ads
[4134:4134:0208/235705.252155:VERBOSE1:ads_service_impl.cc(1769)] Timeout ad notification with uuid db014190-893c-4543-9479-3255206e2e86 in 120 seconds
[4134:4134:0208/235705.252671:VERBOSE1:ad_notification_serving.cc(101)] Ad notification delivered:
  uuid: db014190-893c-4543-9479-3255206e2e86
  creativeInstanceId: 30ecd70f-f56d-4745-a457-4035fcab54b6
  creativeSetId: 0a697a3b-233a-44bc-a337-e2441fb277a7
  campaignId: d4d0474c-277a-4714-b25d-7b484cdb580a
  segment: technology & computing-software
  title: This is a test
  body: This is a test
  targetUrl: https://www.brave.com

Process purchase intent segment and verify sites are extracted

[4748:4748:0208/235931.075227:VERBOSE1:purchase_intent_processor.cc(101)] Extracted purchase intent signal from visited URL

"purchaseIntentSignalHistory":{"AC001":[{"timestamp_in_seconds":1612825171,"weight":1}]}

Process purchase intent segment and verify keywords are extracted

[6014:6014:0209/000651.762424:VERBOSE1:purchase_intent_processor.cc(101)] Extracted purchase intent signal from visited URL

"purchaseIntentSignalHistory":{"BF001":[{"timestamp_in_seconds":1612825609,"weight":1}

Verify that bandit is processing (despite the model being disabled, that is by design)

[6014:6014:0209/001033.366224:VERBOSE1:epsilon_greedy_bandit_processor.cc(118)] Epsilon greedy bandit processed BraveAdsAdNotificationEventType::kDismissed

Verify that the model is not suggesting segments to the ad serve pipeline

Nothing shows up under Exploring... or Exploiting...

Variant models enabled (Epsilon Greedy Bandit and Purchase Intent: --disable-features="TextClassification" --enable-features="EpsilonGreedyBandit")

Used --args --enable-logging=stderr --vmodule="*/bat-native-ledger/*"=6,"*/brave_rewards/*"=6,"*/bat-native-ads/*"=6,"*/bat-native-confirmations/*"=6,"*/brave_ads/*"=6,"*/brave_user_model/*"=6 --brave-ads-debug --disable-features="TextClassification" --enable-features="EpsilonGreedyBandit"

Verify that Epsilon Greedy Bandit and Purchase Intent models are enabled and Text Classification is disabled

[7253:7253:0209/001451.561996:VERBOSE1:features.cc(59)] Text classification feature is disabled
[7253:7253:0209/001451.562154:VERBOSE1:features.cc(61)] Epsilon greedy bandit feature is enabled
[7253:7253:0209/001451.562343:VERBOSE1:features.cc(63)] Purchase intent feature is enabled

Process purchase intent segment and verify keywords are extracted

[7253:7253:0209/001642.064751:VERBOSE1:purchase_intent_processor.cc(101)] Extracted purchase intent signal from visited URL

"purchaseIntentSignalHistory":{"Mortgage and Refinance":[{"timestamp_in_seconds":1612826202,"weight":1}

Verify that bandit is processing

[7253:7253:0209/001451.287621:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for family & parenting segment
[7253:7253:0209/001451.287707:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for fashion segment
[7253:7253:0209/001451.287801:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for folklore segment
[7253:7253:0209/001451.287887:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for food & drink segment
[7253:7253:0209/001451.288014:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for health & fitness segment
[7253:7253:0209/001451.288100:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for history segment
[7253:7253:0209/001451.288194:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for hobbies & interests segment
[7253:7253:0209/001451.288279:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for home segment
[7253:7253:0209/001451.288373:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for law segment
[7253:7253:0209/001451.288489:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for military segment
[7253:7253:0209/001451.288586:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for personal finance segment
[7253:7253:0209/001451.288671:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for pets segment
[7253:7253:0209/001451.288765:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for politics segment
[7253:7253:0209/001451.288851:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for real estate segment
[7253:7253:0209/001451.289155:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for religion segment
[7253:7253:0209/001451.289262:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for science segment
[7253:7253:0209/001451.289348:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for society segment
[7253:7253:0209/001451.289497:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for sports segment
[7253:7253:0209/001451.289584:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for technology & computing segment
[7253:7253:0209/001451.289678:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for travel segment
[7253:7253:0209/001451.289763:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for weather segment
[7253:7253:0209/001451.289858:VERBOSE2:epsilon_greedy_bandit_processor.cc(51)] Epsilon greedy bandit invalid arm was reset for crypto segment

Verify that the model is suggesting segments to the ad serve pipeline

[7253:7253:0209/001630.676907:VERBOSE2:epsilon_greedy_bandit_model.cc(165)] Exploiting epsilon greedy bandit segments:
[7253:7253:0209/001630.676950:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   hobbies & interests
[7253:7253:0209/001630.677069:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   business
[7253:7253:0209/001630.677102:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   technology & computing

Restart after one round of training and make sure state wasn't deleted in user Preferences JSON key epsilon_greedy_bandit_arms

In addition to the details below, I verified that the pulls and values data persisted between sessions/restarts of the browser, and were accurately reflected via brave://rewards 7-day History panel.

First technology & computing-software ad clicked

[9702:9702:0209/003830.104953:VERBOSE1:ads_impl.cc(186)] Browser state changed to unidle
[9702:9702:0209/003830.105835:VERBOSE2:epsilon_greedy_bandit_model.cc(165)] Exploiting epsilon greedy bandit segments:
[9702:9702:0209/003830.105881:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   technology & computing
[9702:9702:0209/003830.109432:VERBOSE1:ad_notification_serving.cc(165)] Serve ad for segments:
[9702:9702:0209/003830.109474:VERBOSE1:ad_notification_serving.cc(167)]   technology & computing
[9702:9702:0209/003830.128211:VERBOSE2:ad_notification_serving.cc(273)] 2 eligible ads before pacing
[9702:9702:0209/003830.128261:VERBOSE2:eligible_ads_priority_filter.cc(81)] 2 eligible ads with a priority of 1
[9702:9702:0209/003830.128302:VERBOSE2:ad_notification_serving.cc(278)] 2 eligible ads after pacing
[9702:9702:0209/003830.128322:VERBOSE1:ad_notification_serving.cc(264)] Found 2 eligible ads
[9702:9702:0209/003830.130100:VERBOSE1:ads_service_impl.cc(1769)] Timeout ad notification with uuid a7e8bd1b-5c36-4258-9312-446075ba4eda in 120 seconds
[9702:9702:0209/003830.130191:VERBOSE1:ad_notification_serving.cc(101)] Ad notification delivered:
  uuid: a7e8bd1b-5c36-4258-9312-446075ba4eda
  creativeInstanceId: ed6c467a-8643-42eb-a7cd-32e878fa3cd4
  creativeSetId: a9f0b49c-3298-4c1b-a8b5-76a71e8d7332
  campaignId: 0e70c893-b0c0-4dab-860d-7e65446f0f0f
  segment: technology & computing
  title: This is a test 1
  body: This is a test 1
  targetUrl: https://www.brave.com

After ad is clicked the value is increased to 1
{\"pulls\":1,\"segment\":\"technology & computing\",\"value\":1.0}

"epsilon_greedy_bandit_arms": "{\"architecture\":{\"pulls\":0,\"segment\":\"architecture\",\"value\":1.0},\"arts & entertainment\":{\"pulls\":0,\"segment\":\"arts & entertainment\",\"value\":1.0},\"automotive\":{\"pulls\":0,\"segment\":\"automotive\",\"value\":1.0},\"business\":{\"pulls\":0,\"segment\":\"business\",\"value\":1.0},\"careers\":{\"pulls\":0,\"segment\":\"careers\",\"value\":1.0},\"cell phones\":{\"pulls\":0,\"segment\":\"cell phones\",\"value\":1.0},\"crypto\":{\"pulls\":0,\"segment\":\"crypto\",\"value\":1.0},\"drugs\":{\"pulls\":0,\"segment\":\"drugs\",\"value\":1.0},\"education\":{\"pulls\":0,\"segment\":\"education\",\"value\":1.0},\"family & parenting\":{\"pulls\":0,\"segment\":\"family & parenting\",\"value\":1.0},\"fashion\":{\"pulls\":0,\"segment\":\"fashion\",\"value\":1.0},\"folklore\":{\"pulls\":0,\"segment\":\"folklore\",\"value\":1.0},\"food & drink\":{\"pulls\":0,\"segment\":\"food & drink\",\"value\":1.0},\"health & fitness\":{\"pulls\":0,\"segment\":\"health & fitness\",\"value\":1.0},\"history\":{\"pulls\":0,\"segment\":\"history\",\"value\":1.0},\"hobbies & interests\":{\"pulls\":0,\"segment\":\"hobbies & interests\",\"value\":1.0},\"home\":{\"pulls\":0,\"segment\":\"home\",\"value\":1.0},\"law\":{\"pulls\":0,\"segment\":\"law\",\"value\":1.0},\"military\":{\"pulls\":0,\"segment\":\"military\",\"value\":1.0},\"personal finance\":{\"pulls\":0,\"segment\":\"personal finance\",\"value\":1.0},\"pets\":{\"pulls\":0,\"segment\":\"pets\",\"value\":1.0},\"politics\":{\"pulls\":0,\"segment\":\"politics\",\"value\":1.0},\"real estate\":{\"pulls\":0,\"segment\":\"real estate\",\"value\":1.0},\"religion\":{\"pulls\":0,\"segment\":\"religion\",\"value\":1.0},\"science\":{\"pulls\":0,\"segment\":\"science\",\"value\":1.0},\"society\":{\"pulls\":0,\"segment\":\"society\",\"value\":1.0},\"sports\":{\"pulls\":0,\"segment\":\"sports\",\"value\":1.0},\"technology & computing\":{\"pulls\":1,\"segment\":\"technology & computing\",\"value\":1.0},\"travel\":{\"pulls\":0,\"segment\":\"travel\",\"value\":1.0},\"weather\":{\"pulls\":0,\"segment\":\"weather\",\"value\":1.0}}",
"epsilon_greedy_bandit_eligible_segments": "[\"technology & computing\"]",

Second ad was dismissed

[10393:10393:0209/005930.100768:VERBOSE1:ads_impl.cc(186)] Browser state changed to unidle
[10393:10393:0209/005930.103106:VERBOSE2:epsilon_greedy_bandit_model.cc(165)] Exploiting epsilon greedy bandit segments:
[10393:10393:0209/005930.103179:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   technology & computing
[10393:10393:0209/005930.108020:VERBOSE1:ad_notification_serving.cc(165)] Serve ad for segments:
[10393:10393:0209/005930.108067:VERBOSE1:ad_notification_serving.cc(167)]   technology & computing
[10393:10393:0209/005930.109104:VERBOSE1:eligible_ad_notifications.cc(70)] All advertisers have been shown, so round robin
[10393:10393:0209/005930.109181:VERBOSE1:client.cc(385)] Resetting seen advertisers
[10393:10393:0209/005930.111029:VERBOSE2:ad_notification_serving.cc(273)] 1 eligible ads before pacing
[10393:10393:0209/005930.111072:VERBOSE2:eligible_ads_priority_filter.cc(81)] 1 eligible ads with a priority of 1
[10393:10393:0209/005930.111116:VERBOSE2:ad_notification_serving.cc(278)] 1 eligible ads after pacing
[10393:10393:0209/005930.111170:VERBOSE1:ad_notification_serving.cc(264)] Found 1 eligible ads
[10393:10393:0209/005930.113991:VERBOSE1:ads_service_impl.cc(1769)] Timeout ad notification with uuid 65b35e8c-9730-4c3d-ac8c-8dc1f2367062 in 120 seconds
[10393:10393:0209/005930.114175:VERBOSE1:ad_notification_serving.cc(101)] Ad notification delivered:
  uuid: 65b35e8c-9730-4c3d-ac8c-8dc1f2367062
  creativeInstanceId: 30ecd70f-f56d-4745-a457-4035fcab54b6
  creativeSetId: 0a697a3b-233a-44bc-a337-e2441fb277a7
  campaignId: d4d0474c-277a-4714-b25d-7b484cdb580a
  segment: technology & computing
  title: This is a test 2
  body: This is a test 2
  targetUrl: https://www.brave.com
[10393:10393:0209/010030.639697:VERBOSE6:ad_notification_event_dismissed.cc(36)] Successfully logged ad notification dismissed event

After ad is dismissed the value is decreased to 0.5
{\"pulls\":2,\"segment\":\"technology & computing\",\"value\":0.5}

"epsilon_greedy_bandit_arms": "{\"architecture\":{\"pulls\":0,\"segment\":\"architecture\",\"value\":1.0},\"arts & entertainment\":{\"pulls\":0,\"segment\":\"arts & entertainment\",\"value\":1.0},\"automotive\":{\"pulls\":0,\"segment\":\"automotive\",\"value\":1.0},\"business\":{\"pulls\":0,\"segment\":\"business\",\"value\":1.0},\"careers\":{\"pulls\":0,\"segment\":\"careers\",\"value\":1.0},\"cell phones\":{\"pulls\":0,\"segment\":\"cell phones\",\"value\":1.0},\"crypto\":{\"pulls\":0,\"segment\":\"crypto\",\"value\":1.0},\"drugs\":{\"pulls\":0,\"segment\":\"drugs\",\"value\":1.0},\"education\":{\"pulls\":0,\"segment\":\"education\",\"value\":1.0},\"family & parenting\":{\"pulls\":0,\"segment\":\"family & parenting\",\"value\":1.0},\"fashion\":{\"pulls\":0,\"segment\":\"fashion\",\"value\":1.0},\"folklore\":{\"pulls\":0,\"segment\":\"folklore\",\"value\":1.0},\"food & drink\":{\"pulls\":0,\"segment\":\"food & drink\",\"value\":1.0},\"health & fitness\":{\"pulls\":0,\"segment\":\"health & fitness\",\"value\":1.0},\"history\":{\"pulls\":0,\"segment\":\"history\",\"value\":1.0},\"hobbies & interests\":{\"pulls\":0,\"segment\":\"hobbies & interests\",\"value\":1.0},\"home\":{\"pulls\":0,\"segment\":\"home\",\"value\":1.0},\"law\":{\"pulls\":0,\"segment\":\"law\",\"value\":1.0},\"military\":{\"pulls\":0,\"segment\":\"military\",\"value\":1.0},\"personal finance\":{\"pulls\":0,\"segment\":\"personal finance\",\"value\":1.0},\"pets\":{\"pulls\":0,\"segment\":\"pets\",\"value\":1.0},\"politics\":{\"pulls\":0,\"segment\":\"politics\",\"value\":1.0},\"real estate\":{\"pulls\":0,\"segment\":\"real estate\",\"value\":1.0},\"religion\":{\"pulls\":0,\"segment\":\"religion\",\"value\":1.0},\"science\":{\"pulls\":0,\"segment\":\"science\",\"value\":1.0},\"society\":{\"pulls\":0,\"segment\":\"society\",\"value\":1.0},\"sports\":{\"pulls\":0,\"segment\":\"sports\",\"value\":1.0},\"technology & computing\":{\"pulls\":2,\"segment\":\"technology & computing\",\"value\":0.5},\"travel\":{\"pulls\":0,\"segment\":\"travel\",\"value\":1.0},\"weather\":{\"pulls\":0,\"segment\":\"weather\",\"value\":1.0}}",
"epsilon_greedy_bandit_eligible_segments": "[\"technology & computing\"]",

GeetaSarvadnya · 2021-02-08T17:34:41Z

Verification passed on

Brave | 1.20.100 Chromium: 88.0.4324.146 (Official Build) (64-bit)
-- | --
Revision | 406dc88511162d6598242f2c709be1414a042fb0-refs/branch-heads/4324@{#2088}
OS | Windows 10 OS Version 2004 (Build 19041.746)

Verified the test plan from Add eligible segment filter for bandits brave-core#7533

Case 1: Default models enabled

Verified that Text Classification and Purchase Intent models are enabled and Epsilon Greedy Bandit is disabled by default

[18372:19208:0208/193551.986:VERBOSE1:features.cc(59)] Text classification feature is enabled
[18372:19208:0208/193551.987:VERBOSE1:features.cc(61)] Epsilon greedy bandit feature is disabled
[18372:19208:0208/193551.988:VERBOSE1:features.cc(63)] Purchase intent feature is enabled

Viewed an AD and ensured ads are triggered via text classification

[15184:11084:0209/002915.741:VERBOSE1:text_classification_processor.cc(62)] Classified text with the top segment as personal finance-personal finance
[15184:11084:0209/002915.747:VERBOSE1:conversions.cc(151)] No conversions found for visited URL
[15184:11084:0209/002928.228:VERBOSE1:ads_impl.cc(178)] Browser state changed to idle
[15184:11084:0209/002944.331:VERBOSE1:ads_impl.cc(186)] Browser state changed to unidle
[15184:11084:0209/002944.368:VERBOSE1:ad_notification_serving.cc(165)] Serve ad for segments:
[15184:11084:0209/002944.368:VERBOSE1:ad_notification_serving.cc(167)]   personal finance-personal finance
[15184:11084:0209/002944.373:VERBOSE1:ad_notification_serving.cc(167)]   food & drink-cooking
[15184:11084:0209/002944.381:VERBOSE1:ad_notification_serving.cc(167)]   business-marketing
[15184:11084:0209/002944.384:VERBOSE1:ad_notification_serving.cc(182)] No eligible ads found for segments
[15184:11084:0209/002944.384:VERBOSE1:ad_notification_serving.cc(197)] Serve ad for parent segments:
[15184:11084:0209/002944.384:VERBOSE1:ad_notification_serving.cc(199)]   personal finance
[15184:11084:0209/002944.385:VERBOSE1:ad_notification_serving.cc(199)]   food & drink
[15184:11084:0209/002944.386:VERBOSE1:ad_notification_serving.cc(199)]   business
[15184:11084:0209/002944.387:VERBOSE1:ad_notification_serving.cc(214)] No eligible ads found for parent segments
[15184:11084:0209/002944.388:VERBOSE1:ad_notification_serving.cc(226)] Serve untargeted ad
[15184:11084:0209/002944.390:VERBOSE2:ad_notification_serving.cc(273)] 3 eligible ads before pacing
[15184:11084:0209/002944.390:VERBOSE2:eligible_ads_priority_filter.cc(81)] 3 eligible ads with a priority of 1
[15184:11084:0209/002944.391:VERBOSE2:ad_notification_serving.cc(278)] 3 eligible ads after pacing
[15184:11084:0209/002944.391:VERBOSE1:ad_notification_serving.cc(264)] Found 3 eligible ads
[15184:11084:0209/002944.392:VERBOSE1:ads_service_impl.cc(1769)] Timeout ad notification with uuid 3a4e401f-d470-4dda-b70b-6e1c29d54b2a in 120 seconds
[15184:11084:0209/002944.392:VERBOSE1:ad_notification_serving.cc(101)] Ad notification delivered:
  uuid: 3a4e401f-d470-4dda-b70b-6e1c29d54b2a
  creativeInstanceId: 30ecd70f-f56d-4745-a457-4035fcab54b6
  creativeSetId: 0a697a3b-233a-44bc-a337-e2441fb277a7
  campaignId: d4d0474c-277a-4714-b25d-7b484cdb580a
  segment: untargeted
  title: This is a test
  body: This is a test
  targetUrl: https://www.brave.com
[15184:11084:0209/002944.393:VERBOSE3:ad_notification_event_viewed.cc(24)] Viewed ad notification with uuid 3a4e401f-d470-4dda-b70b-6e1c29d54b2a and creative instance id 30ecd70f-f56d-4745-a457-4035fcab54b6
[15184:11084:0209/002944.402:VERBOSE1:confirmations.cc(90)] Confirming view ad for creative instance id 30ecd70f-f56d-4745-a457-4035fcab54b6

Viewed an AD using the search keyword coinbase earn and ensured purchase intent for Keywords are working as expected

[14368:16036:0209/001816.206:VERBOSE1:purchase_intent_resource.cc(58)] Successfully loaded kkjipiepeooghlclkedllogndmohhnhi purchase intent resource
[14368:16036:0209/001816.208:VERBOSE1:purchase_intent_resource.cc(235)] Parsed purchase intent user model version 1
[14368:16036:0209/001816.208:VERBOSE1:purchase_intent_resource.cc(68)] Successfully initialized kkjipiepeooghlclkedllogndmohhnhi purchase intent resource

[14368:16036:0209/001842.627:VERBOSE1:purchase_intent_processor.cc(101)] Extracted purchase intent signal from visited URL
[14368:16036:0209/001842.628:VERBOSE1:ads_impl.cc(170)] Search engine pages are not supported for text classification
[14368:16036:0209/001842.637:VERBOSE1:conversions.cc(151)] No conversions found for visited URL
[14368:16036:0209/001843.100:VERBOSE2:tab_manager.cc(93)] Tab id 4 was updated
[14368:16036:0209/001843.105:VERBOSE1:conversions.cc(130)] Checking URL for conversions
[14368:16036:0209/001843.114:VERBOSE1:purchase_intent_processor.cc(101)] Extracted purchase intent signal from visited URL
[14368:16036:0209/001843.115:VERBOSE1:ads_impl.cc(170)] Search engine pages are not supported for text classification
[14368:16036:0209/001843.126:VERBOSE1:conversions.cc(151)] No conversions found for visited URL
[14368:16036:0209/001857.123:VERBOSE1:ads_impl.cc(178)] Browser state changed to idle
[14368:16036:0209/001911.246:VERBOSE1:ads_impl.cc(186)] Browser state changed to unidle
[14368:16036:0209/001911.249:VERBOSE1:text_classification_model.cc(94)] No text classification probabilities found for en-US locale
[14368:16036:0209/001911.262:VERBOSE1:ad_notification_serving.cc(165)] Serve ad for segments:
[14368:16036:0209/001911.262:VERBOSE1:ad_notification_serving.cc(167)]   untargeted
[14368:16036:0209/001911.265:VERBOSE2:ad_notification_serving.cc(273)] 3 eligible ads before pacing
[14368:16036:0209/001911.265:VERBOSE2:eligible_ads_priority_filter.cc(81)] 3 eligible ads with a priority of 1
[14368:16036:0209/001911.265:VERBOSE2:ad_notification_serving.cc(278)] 3 eligible ads after pacing
[14368:16036:0209/001911.273:VERBOSE1:ad_notification_serving.cc(264)] Found 3 eligible ads
[14368:16036:0209/001911.274:VERBOSE1:ads_service_impl.cc(1769)] Timeout ad notification with uuid 4ae50fd1-ab26-4422-a539-bc4d36f919ce in 120 seconds
[14368:16036:0209/001911.274:VERBOSE1:ad_notification_serving.cc(101)] Ad notification delivered:
  uuid: 4ae50fd1-ab26-4422-a539-bc4d36f919ce
  creativeInstanceId: 4983dc92-09fd-4e5c-94c3-616a4728c1d9
  creativeSetId: 5bdeab83-048f-48a7-9602-a1092ded123c
  campaignId: 63010037-c230-40db-b41a-3127619b88e4
  segment: untargeted
  title: This is a test
  body: This is a test
  targetUrl: https://www.brave.com
[14368:16036:0209/001911.275:VERBOSE3:ad_notification_event_viewed.cc(24)] Viewed ad notification with uuid 4ae50fd1-ab26-4422-a539-bc4d36f919ce and creative instance id 4983dc92-09fd-4e5c-94c3-616a4728c1d9
[14368:16036:0209/001911.275:VERBOSE1:confirmations.cc(90)] Confirming view ad for creative instance id 4983dc92-09fd-4e5c-94c3-616a4728c1d9

Ensured Epsilon greedy bandit is processed (despite the model being disabled, that is by design)

[14368:16036:0209/001913.532:VERBOSE1:epsilon_greedy_bandit_processor.cc(118)] Epsilon greedy bandit processed BraveAdsAdNotificationEventType::kClicked
[14368:16036:0209/001913.541:VERBOSE1:ad_notification_event_clicked.cc(36)] Successfully logged ad notification clicked event

Ensured model is not suggesting segments to the ad serve pipeline and didn't see Exploring epsilon greedy bandit segments: ... or Exploiting epsilon greedy bandit segments: ...

Case 2: Variant models enabled

Verified that Purchase Intent models and Epsilon Greedy Bandit are enabled and Text Classification is disabled when the brave is launched using cmd line: --enable-logging=stderr --vmodule="*/bat-native-ads/*"=9,"*/brave_ads/*"=9,"*/brave_user_model/*"=9,"*/bat_ads/*"=9 --brave-ads-staging --rewards=staging=true --disable-features="TextClassification" --enable-features="EpsilonGreedyBandit"

[2292:8648:0208/232707.461:VERBOSE1:features.cc(59)] Text classification feature is disabled
[2292:8648:0208/232707.462:VERBOSE1:features.cc(61)] Epsilon greedy bandit feature is enabled
[2292:8648:0208/232707.463:VERBOSE1:features.cc(63)] Purchase intent feature is enabled

Viewed an AD using the search keyword best bitcoin lending sites and ensured purchase intent for Keywords are working as expected

[17692:6200:0209/003226.279:VERBOSE1:purchase_intent_resource.cc(58)] Successfully loaded kkjipiepeooghlclkedllogndmohhnhi purchase intent resource
[17692:6200:0209/003226.281:VERBOSE1:purchase_intent_resource.cc(235)] Parsed purchase intent user model version 1
[17692:6200:0209/003226.292:VERBOSE1:purchase_intent_resource.cc(68)] Successfully initialized kkjipiepeooghlclkedllogndmohhnhi purchase intent resource

[17692:6200:0209/003312.822:VERBOSE1:purchase_intent_processor.cc(101)] Extracted purchase intent signal from visited URL
[17692:6200:0209/003312.823:VERBOSE9:client.cc(455)] Saving client state
[17692:6200:0209/003312.823:VERBOSE9:client.cc(455)] Saving client state
[17692:6200:0209/003312.834:VERBOSE1:ads_impl.cc(170)] Search engine pages are not supported for text classification
[17692:6200:0209/003312.837:VERBOSE9:client.cc(470)] Successfully saved client state
[17692:6200:0209/003312.841:VERBOSE9:client.cc(470)] Successfully saved client state
[17692:6200:0209/003312.846:VERBOSE1:conversions.cc(151)] No conversions found for visited URL
[17692:6200:0209/003313.285:VERBOSE2:tab_manager.cc(93)] Tab id 4 was updated
[17692:6200:0209/003313.287:VERBOSE1:conversions.cc(130)] Checking URL for conversions
[17692:6200:0209/003313.291:VERBOSE1:purchase_intent_processor.cc(101)] Extracted purchase intent signal from visited URL
[17692:6200:0209/003313.292:VERBOSE9:client.cc(455)] Saving client state
[17692:6200:0209/003313.296:VERBOSE9:client.cc(455)] Saving client state
[17692:6200:0209/003313.297:VERBOSE1:ads_impl.cc(170)] Search engine pages are not supported for text classification
[17692:6200:0209/003313.315:VERBOSE9:client.cc(470)] Successfully saved client state
[17692:6200:0209/003313.323:VERBOSE9:client.cc(470)] Successfully saved client state
[17692:6200:0209/003313.323:VERBOSE1:conversions.cc(151)] No conversions found for visited URL
[17692:6200:0209/003327.824:VERBOSE1:ads_impl.cc(178)] Browser state changed to idle
[17692:6200:0209/003341.933:VERBOSE1:ads_impl.cc(186)] Browser state changed to unidle
[17692:6200:0209/003341.936:VERBOSE2:epsilon_greedy_bandit_model.cc(165)] Exploiting epsilon greedy bandit segments:
[17692:6200:0209/003341.947:INFO:notification_helper_win.cc(171)] Focus Assist is disabled
[17692:6200:0209/003341.954:INFO:notification_helper_win.cc(208)] Notifications are enabled
[17692:6200:0209/003341.961:VERBOSE1:ad_notification_serving.cc(160)] No segments to serve targeted ads
[17692:6200:0209/003341.961:VERBOSE1:ad_notification_serving.cc(226)] Serve untargeted ad
[17692:6200:0209/003341.967:VERBOSE2:ad_notification_serving.cc(273)] 3 eligible ads before pacing
[17692:6200:0209/003341.967:VERBOSE2:eligible_ads_priority_filter.cc(81)] 3 eligible ads with a priority of 1
[17692:6200:0209/003341.968:VERBOSE2:ad_notification_serving.cc(278)] 3 eligible ads after pacing
[17692:6200:0209/003341.968:VERBOSE1:ad_notification_serving.cc(264)] Found 3 eligible ads
[17692:6200:0209/003341.969:VERBOSE9:client.cc(455)] Saving client state
[17692:6200:0209/003341.971:VERBOSE9:ad_notifications.cc(390)] Saving ad notifications state
[17692:6200:0209/003341.972:VERBOSE1:ads_service_impl.cc(1769)] Timeout ad notification with uuid 62c10745-36b3-44b5-b5e1-9f871f425be8 in 120 seconds
[17692:6200:0209/003341.981:VERBOSE9:client.cc(455)] Saving client state
[17692:6200:0209/003341.981:VERBOSE1:ad_notification_serving.cc(101)] Ad notification delivered:
  uuid: 62c10745-36b3-44b5-b5e1-9f871f425be8
  creativeInstanceId: ed6c467a-8643-42eb-a7cd-32e878fa3cd4
  creativeSetId: a9f0b49c-3298-4c1b-a8b5-76a71e8d7332
  campaignId: 0e70c893-b0c0-4dab-860d-7e65446f0f0f
  segment: untargeted
  title: This is a test
  body: This is a test
  targetUrl: https://www.brave.com
[17692:6200:0209/003341.989:VERBOSE3:ad_notification_event_viewed.cc(24)] Viewed ad notification with uuid 62c10745-36b3-44b5-b5e1-9f871f425be8 and creative instance id ed6c467a-8643-42eb-a7cd-32e878fa3cd4

Ensured Epsilon greedy bandit is processed (despite the model being disabled, that is by design)

[17692:6200:0209/003344.077:VERBOSE1:epsilon_greedy_bandit_processor.cc(118)] Epsilon greedy bandit processed BraveAdsAdNotificationEventType::kClicked
[17692:6200:0209/003344.082:VERBOSE9:ad_notifications.cc(405)] Successfully saved ad notifications state
[17692:6200:0209/003344.083:VERBOSE1:ad_notification_event_clicked.cc(36)] Successfully logged ad notification clicked event

Verified that the model is suggesting segments to the ad serve pipeline Exploring epsilon greedy bandit segments: ... or Exploiting epsilon greedy bandit segments: ...

[17692:6200:0209/003341.936:VERBOSE2:epsilon_greedy_bandit_model.cc(165)] Exploiting epsilon greedy bandit segments:

Verified that the pulls and values data persisted between sessions/restarts of the browser, and were accurately reflected via brave://rewards 7-day History panel.

Brave | 1.20.101 Chromium: 88.0.4324.152 (Official Build) (64-bit)
-- | --
Revision | 6579930fc53b4dc589c042bec9d0a3778326974d-refs/branch-heads/4324@{#2106}
OS | Windows 10 OS Version 2004 (Build 19041.746)

Case 3: Restart after one round of training and make sure state wasn't deleted in user Preferences JSON key epsilon_greedy_bandit_arms

Verified the following checks in 3.x cases to ensure Case 3 is working as expected
- Default: pull 0 value 1.0
- Click on AD: pull 1 value 1.0
- Dismiss an AD: pull 1 value 0.0
- Time Out: Values are incorrect - logged epsilon_greedy_bandit_arms isn't displaying expected values for Brave Ads time out #14054
- View 1st Ad and dismiss 2nd Ad - pull value is 2 and value is 0.5
- Click two Ads of same segment - pull becomes 2 and value is 1.0
- Dismiss two Ads of same segment - pull becomes 2 and value is 0.0

Case 3.1: Verified default pulls values is ` 0` and value is `1.0` in the Preference file for all the segments

Verified that the default value pulls is set to 0 and value is set to 1.0 in the Preference file when Rewards is enabled

"epsilon_greedy_bandit_arms": "{\"architecture\":{\"pulls\":0,\"segment\":\"architecture\",\"value\":1.0},
                                      \"arts & entertainment\":{\"pulls\":0,\"segment\":\"arts & entertainment\",\"value\":1.0},
                                      \"automotive\":{\"pulls\":0,\"segment\":\"automotive\",\"value\":1.0},
                                      \"business\":{\"pulls\":0,\"segment\":\"business\",\"value\":1.0},
                                      \"careers\":{\"pulls\":0,\"segment\":\"careers\",\"value\":1.0},
                                      \"cell phones\":{\"pulls\":0,\"segment\":\"cell phones\",\"value\":1.0},
                                      \"crypto\":{\"pulls\":0,\"segment\":\"crypto\",\"value\":1.0},
                                      \"drugs\":{\"pulls\":0,\"segment\":\"drugs\",\"value\":1.0},
                                      \"education\":{\"pulls\":0,\"segment\":\"education\",\"value\":1.0},
                                      \"family & parenting\":{\"pulls\":0,\"segment\":
                                      \"family & parenting\",\"value\":1.0},
                                      \"fashion\":{\"pulls\":0,\"segment\":\"fashion\",\"value\":1.0},
                                      \"folklore\":{\"pulls\":0,\"segment\":\"folklore\",\"value\":1.0},
                                      \"food & drink\":{\"pulls\":0,\"segment\":\"food & drink\",\"value\":1.0},
                                      \"health & fitness\":{\"pulls\":0,\"segment\":\"health & fitness\",\"value\":1.0},
                                      \"history\":{\"pulls\":0,\"segment\":\"history\",\"value\":1.0},
                                      \"hobbies & interests\":{\"pulls\":0,\"segment\":\"hobbies & interests\",\"value\":1.0},
                                      \"home\":{\"pulls\":0,\"segment\":\"home\",\"value\":1.0},
                                      \"law\":{\"pulls\":0,\"segment\":\"law\",\"value\":1.0},
                                      \"military\":{\"pulls\":0,\"segment\":\"military\",\"value\":1.0},
                                      \"personal finance\":{\"pulls\":0,\"segment\":\"personal finance\",\"value\":1.0},
                                      \"pets\":{\"pulls\":0,\"segment\":\"pets\",\"value\":1.0},
                                      \"politics\":{\"pulls\":0,\"segment\":\"politics\",\"value\":1.0},
                                      \"real estate\":{\"pulls\":0,\"segment\":\"real estate\",\"value\":1.0},
                                      \"religion\":{\"pulls\":0,\"segment\":\"religion\",\"value\":1.0},
                                      \"science\":{\"pulls\":0,\"segment\":\"science\",\"value\":1.0},
                                      \"society\":{\"pulls\":0,\"segment\":\"society\",\"value\":1.0},
                                      \"sports\":{\"pulls\":0,\"segment\":\"sports\",\"value\":1.0},
                                      \"technology & computing\":{\"pulls\":0,\"segment\":\"technology & computing\",\"value\":1.0},
                                      \"travel\":{\"pulls\":0,\"segment\":\"travel\",\"value\":1.0},
                                      \"weather\":{\"pulls\":0,\"segment\":\"weather\",\"value\":1.0}}",

Case 3.2: Clicked on AD and verified `pulls` values becomes `1` and `value` remained `1.0` in the Preference file - military segment

Clicked on AD:

[16324:4816:0209/141406.890:VERBOSE2:epsilon_greedy_bandit_model.cc(165)] Exploiting epsilon greedy bandit segments:
[16324:4816:0209/141406.890:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   military
[16324:4816:0209/141406.897:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   arts & entertainment
[16324:4816:0209/141406.898:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   business
[16324:4816:0209/141406.899:INFO:notification_helper_win.cc(171)] Focus Assist is disabled
[16324:4816:0209/141406.911:INFO:notification_helper_win.cc(208)] Notifications are enabled
[16324:4816:0209/141406.913:VERBOSE1:ad_notification_serving.cc(165)] Serve ad for segments:
[16324:4816:0209/141406.917:VERBOSE1:ad_notification_serving.cc(167)]   military
[16324:4816:0209/141406.917:VERBOSE1:ad_notification_serving.cc(167)]   arts & entertainment
[16324:4816:0209/141406.918:VERBOSE1:ad_notification_serving.cc(167)]   business
[16324:4816:0209/141406.925:VERBOSE2:ad_notification_serving.cc(273)] 17 eligible ads before pacing
[16324:4816:0209/141406.925:VERBOSE2:ad_notification_pacing.cc(46)] Pacing ad delivery for creative instance id afa1c8c5-18f8-408a-81ef-d8e424ccd76c [Roll(0.5):0.716005]
[16324:4816:0209/141406.925:VERBOSE2:eligible_ads_priority_filter.cc(81)] 16 eligible ads with a priority of 1
[16324:4816:0209/141406.933:VERBOSE2:ad_notification_serving.cc(278)] 16 eligible ads after pacing
[16324:4816:0209/141406.933:VERBOSE1:ad_notification_serving.cc(264)] Found 16 eligible ads
[16324:4816:0209/141406.934:VERBOSE1:ads_service_impl.cc(1769)] Timeout ad notification with uuid 83327656-fe1c-423e-b1f2-2796e3b6a3dc in 120 seconds
[16324:4816:0209/141406.934:VERBOSE1:ad_notification_serving.cc(101)] Ad notification delivered:
  uuid: 83327656-fe1c-423e-b1f2-2796e3b6a3dc
  creativeInstanceId: 5a3c30d1-2d4e-4de2-82bb-2bac646e9e23
  creativeSetId: 6b869181-7330-4b9c-80b7-d33c50493b22
  campaignId: 37d14e05-fa1f-4730-8fd0-fa44a5ea41b3
  segment: military
  title: Gala Games: $1.6m NFT! ≡ƒöÑ≡ƒöÑ
  body: Want to find out why we sold an NFT for $1.6m?
  targetUrl: https://gala.games/gala-games-and-brave-present-mirandus-shops/
[16324:4816:0209/141406.938:VERBOSE3:ad_notification_event_viewed.cc(24)] Viewed ad notification with uuid 83327656-fe1c-423e-b1f2-2796e3b6a3dc and creative instance id 5a3c30d1-2d4e-4de2-82bb-2bac646e9e23
[16324:4816:0209/141406.944:VERBOSE1:confirmations.cc(90)] Confirming view ad for creative instance id 5a3c30d1-2d4e-4de2-82bb-2bac646e9e23

Ensured that pulls value changed to 1 and value remained 1.0 for click on AD

Case 3.3: Clicked on AD and verified `pulls` value becomes `1` and `value` remained `1.0` in the Preference file - hobbies and interests segment

Clicked on AD:

[11668:4828:0209/143039.326:VERBOSE2:epsilon_greedy_bandit_model.cc(165)] Exploiting epsilon greedy bandit segments:
[11668:4828:0209/143039.327:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   technology & computing
[11668:4828:0209/143039.327:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   science
[11668:4828:0209/143039.337:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   hobbies & interests
[11668:4828:0209/143039.339:INFO:notification_helper_win.cc(171)] Focus Assist is disabled
[11668:4828:0209/143039.345:INFO:notification_helper_win.cc(208)] Notifications are enabled
[11668:4828:0209/143039.346:VERBOSE1:ad_notification_serving.cc(165)] Serve ad for segments:
[11668:4828:0209/143039.346:VERBOSE1:ad_notification_serving.cc(167)]   technology & computing
[11668:4828:0209/143039.353:VERBOSE1:ad_notification_serving.cc(167)]   science
[11668:4828:0209/143039.356:VERBOSE1:ad_notification_serving.cc(167)]   hobbies & interests
[11668:4828:0209/143039.386:VERBOSE2:ad_notification_serving.cc(273)] 28 eligible ads before pacing
[11668:4828:0209/143039.386:VERBOSE2:ad_notification_pacing.cc(46)] Pacing ad delivery for creative instance id 98a93e60-6ede-44f7-85b7-51d6994f5f4d [Roll(0.5):0.608456]
[11668:4828:0209/143039.387:VERBOSE2:eligible_ads_priority_filter.cc(81)] 27 eligible ads with a priority of 1
[11668:4828:0209/143039.394:VERBOSE2:ad_notification_serving.cc(278)] 27 eligible ads after pacing
[11668:4828:0209/143039.396:VERBOSE1:ad_notification_serving.cc(264)] Found 27 eligible ads
[11668:4828:0209/143039.397:VERBOSE1:ads_service_impl.cc(1769)] Timeout ad notification with uuid f00da124-73c8-4acd-8cbc-bfe191a5b080 in 120 seconds
[11668:4828:0209/143039.405:VERBOSE1:ad_notification_serving.cc(101)] Ad notification delivered:
  uuid: f00da124-73c8-4acd-8cbc-bfe191a5b080
  creativeInstanceId: fc702517-2158-46ca-ae19-9c459b0be723
  creativeSetId: 6b869181-7330-4b9c-80b7-d33c50493b22
  campaignId: 37d14e05-fa1f-4730-8fd0-fa44a5ea41b3
  segment: hobbies & interests
  title: Gala Games: Join us & use BAT!
  body: Buy NFTs for the upcoming MMORPG Mirandus with BAT!
  targetUrl: https://gala.games/gala-games-and-brave-present-mirandus-shops/
[11668:4828:0209/143039.405:VERBOSE3:ad_notification_event_viewed.cc(24)] Viewed ad notification with uuid f00da124-73c8-4acd-8cbc-bfe191a5b080 and creative instance id fc702517-2158-46ca-ae19-9c459b0be723
[11668:4828:0209/143039.406:VERBOSE1:confirmations.cc(90)] Confirming view ad for creative instance id fc702517-2158-46ca-ae19-9c459b0be723

Ensured that pulls value changed to 1 and value remained 1.0 for click on AD

Case 3.4: Dismissed an AD and verified that `pulls` value becomes `1` and `value` changed to `0.0` in the Preference file - personal finance segment

Dismissed an AD:

[7108:7452:0209/144509.554:VERBOSE2:epsilon_greedy_bandit_model.cc(165)] Exploiting epsilon greedy bandit segments:
[7108:7452:0209/144509.555:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   business
[7108:7452:0209/144509.556:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   personal finance
[7108:7452:0209/144509.567:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   military
[7108:7452:0209/144509.571:INFO:notification_helper_win.cc(171)] Focus Assist is disabled
[7108:7452:0209/144509.578:INFO:notification_helper_win.cc(208)] Notifications are enabled
[7108:7452:0209/144509.582:VERBOSE1:ad_notification_serving.cc(165)] Serve ad for segments:
[7108:7452:0209/144509.582:VERBOSE1:ad_notification_serving.cc(167)]   business
[7108:7452:0209/144509.582:VERBOSE1:ad_notification_serving.cc(167)]   personal finance
[7108:7452:0209/144509.583:VERBOSE1:ad_notification_serving.cc(167)]   military
[7108:7452:0209/144509.592:VERBOSE2:ad_notification_serving.cc(273)] 24 eligible ads before pacing
[7108:7452:0209/144509.593:VERBOSE2:ad_notification_pacing.cc(46)] Pacing ad delivery for creative instance id 98a93e60-6ede-44f7-85b7-51d6994f5f4d [Roll(0.5):0.83242]
[7108:7452:0209/144509.595:VERBOSE2:ad_notification_pacing.cc(46)] Pacing ad delivery for creative instance id afa1c8c5-18f8-408a-81ef-d8e424ccd76c [Roll(0.5):0.910722]
[7108:7452:0209/144509.595:VERBOSE2:eligible_ads_priority_filter.cc(81)] 22 eligible ads with a priority of 1
[7108:7452:0209/144509.599:VERBOSE2:ad_notification_serving.cc(278)] 22 eligible ads after pacing
[7108:7452:0209/144509.599:VERBOSE1:ad_notification_serving.cc(264)] Found 22 eligible ads
[7108:7452:0209/144509.600:VERBOSE1:ads_service_impl.cc(1769)] Timeout ad notification with uuid e23dfbf7-22aa-4fdd-8229-2f9cf4c79f28 in 120 seconds
[7108:7452:0209/144509.601:VERBOSE1:ad_notification_serving.cc(101)] Ad notification delivered:
  uuid: e23dfbf7-22aa-4fdd-8229-2f9cf4c79f28
  creativeInstanceId: 979735a3-f779-49a2-aa91-59f06f83c1a2
  creativeSetId: 6b869181-7330-4b9c-80b7-d33c50493b22
  campaignId: 37d14e05-fa1f-4730-8fd0-fa44a5ea41b3
  segment: personal finance
  title: Gala Games: Be the First! ≡ƒöÑ≡ƒöÑ
  body: We are spawn-camping old-school gaming! Join us!
  targetUrl: https://gala.games/gala-games-and-brave-present-mirandus-shops/
[7108:7452:0209/144509.604:VERBOSE3:ad_notification_event_viewed.cc(24)] Viewed ad notification with uuid e23dfbf7-22aa-4fdd-8229-2f9cf4c79f28 and creative instance id 979735a3-f779-49a2-aa91-59f06f83c1a2
[7108:7452:0209/144509.611:VERBOSE1:confirmations.cc(90)] Confirming view ad for creative instance id 979735a3-f779-49a2-aa91-59f06f83c1a2

Ensured that pulls value changed to 1 and value changed to 0.0 for dismissing an AD

Case 3.5: Viewed two Ads of same segment/category, click the 1st Ad and dismiss the 2nd AD

Viewed 1st Ad from Technology & Computing segment

[7804:7348:0209/180827.447:VERBOSE2:epsilon_greedy_bandit_model.cc(149)] Exploring epsilon greedy bandit segments:
[7804:7348:0209/180827.447:VERBOSE2:epsilon_greedy_bandit_model.cc(151)]   technology & computing
[7804:7348:0209/180827.457:VERBOSE2:epsilon_greedy_bandit_model.cc(151)]
[7804:7348:0209/180827.458:VERBOSE2:epsilon_greedy_bandit_model.cc(151)]
[7804:7348:0209/180827.467:INFO:notification_helper_win.cc(171)] Focus Assist is disabled
[7804:7348:0209/180827.475:INFO:notification_helper_win.cc(208)] Notifications are enabled
[7804:7348:0209/180827.478:VERBOSE1:ad_notification_serving.cc(165)] Serve ad for segments:
[7804:7348:0209/180827.478:VERBOSE1:ad_notification_serving.cc(167)]   technology & computing
[7804:7348:0209/180827.486:VERBOSE1:ad_notification_serving.cc(167)]
[7804:7348:0209/180827.495:VERBOSE1:ad_notification_serving.cc(167)]
[7804:7348:0209/180827.498:VERBOSE2:ad_notification_serving.cc(273)] 2 eligible ads before pacing
[7804:7348:0209/180827.506:VERBOSE2:eligible_ads_priority_filter.cc(81)] 2 eligible ads with a priority of 1
[7804:7348:0209/180827.507:VERBOSE2:ad_notification_serving.cc(278)] 2 eligible ads after pacing
[7804:7348:0209/180827.507:VERBOSE1:ad_notification_serving.cc(264)] Found 2 eligible ads
[7804:7348:0209/180827.517:VERBOSE1:ads_service_impl.cc(1769)] Timeout ad notification with uuid 4e956a71-b472-4873-a67f-4fbbd1fc3280 in 120 seconds
[7804:7348:0209/180827.518:VERBOSE1:ad_notification_serving.cc(101)] Ad notification delivered:
  uuid: 4e956a71-b472-4873-a67f-4fbbd1fc3280
  creativeInstanceId: 30ecd70f-f56d-4745-a457-4035fcab54b6
  creativeSetId: 0a697a3b-233a-44bc-a337-e2441fb277a7
  campaignId: d4d0474c-277a-4714-b25d-7b484cdb580a
  segment: technology & computing
  title: This is a test 2
  body: This is a test 2
  targetUrl: https://www.brave.com
[7804:7348:0209/180827.522:VERBOSE3:ad_notification_event_viewed.cc(24)] Viewed ad notification with uuid 4e956a71-b472-4873-a67f-4fbbd1fc3280 and creative instance id 30ecd70f-f56d-4745-a457-4035fcab54b6
[7804:7348:0209/180827.529:VERBOSE1:confirmations.cc(90)] Confirming view ad for creative instance id 30ecd70f-f56d-4745-a457-4035fcab54b6
[7804:7348:0209/180827.530:VERBOSE1:redeem_unblinded_token.cc(49)] Redeem unblinded token

Ensured pull value is 1 and value is 1.0

Dismiss 2nd Ad from the same segment

[7804:7348:0209/182103.646:VERBOSE2:epsilon_greedy_bandit_model.cc(165)] Exploiting epsilon greedy bandit segments:
[7804:7348:0209/182103.646:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   technology & computing
[7804:7348:0209/182103.655:INFO:notification_helper_win.cc(171)] Focus Assist is disabled
[7804:7348:0209/182103.661:INFO:notification_helper_win.cc(208)] Notifications are enabled
[7804:7348:0209/182103.663:VERBOSE1:ad_notification_serving.cc(165)] Serve ad for segments:
[7804:7348:0209/182103.663:VERBOSE1:ad_notification_serving.cc(167)]   technology & computing
[7804:7348:0209/182103.670:VERBOSE1:eligible_ad_notifications.cc(70)] All advertisers have been shown, so round robin
[7804:7348:0209/182103.670:VERBOSE1:client.cc(385)] Resetting seen advertisers
[7804:7348:0209/182103.671:VERBOSE2:ad_notification_serving.cc(273)] 1 eligible ads before pacing
[7804:7348:0209/182103.671:VERBOSE2:eligible_ads_priority_filter.cc(81)] 1 eligible ads with a priority of 1
[7804:7348:0209/182103.672:VERBOSE2:ad_notification_serving.cc(278)] 1 eligible ads after pacing
[7804:7348:0209/182103.674:VERBOSE1:ad_notification_serving.cc(264)] Found 1 eligible ads
[7804:7348:0209/182103.674:VERBOSE1:ads_service_impl.cc(1769)] Timeout ad notification with uuid 89924e1e-ff36-4b59-abc8-3f7e2e9071fc in 120 seconds
[7804:7348:0209/182103.675:VERBOSE1:ad_notification_serving.cc(101)] Ad notification delivered:
  uuid: 89924e1e-ff36-4b59-abc8-3f7e2e9071fc
  creativeInstanceId: ed6c467a-8643-42eb-a7cd-32e878fa3cd4
  creativeSetId: a9f0b49c-3298-4c1b-a8b5-76a71e8d7332
  campaignId: 0e70c893-b0c0-4dab-860d-7e65446f0f0f
  segment: technology & computing
  title: This is a test 1
  body: This is a test 1
  targetUrl: https://www.brave.com
[7804:7348:0209/182103.675:VERBOSE3:ad_notification_event_viewed.cc(24)] Viewed ad notification with uuid 89924e1e-ff36-4b59-abc8-3f7e2e9071fc and creative instance id ed6c467a-8643-42eb-a7cd-32e878fa3cd4
[7804:7348:0209/182103.676:VERBOSE1:confirmations.cc(90)] Confirming view ad for creative instance id ed6c467a-8643-42eb-a7cd-32e878fa3cd4

Ensured pull value is 2 and value becomes 0.5

Case 3.6: View an AD don't take any action on the notification wait until AD gets time out

View an AD don't take any action on the Ads notification:

[14444:3632:0209/170103.653:VERBOSE2:epsilon_greedy_bandit_model.cc(165)] Exploiting epsilon greedy bandit segments:
[14444:3632:0209/170103.653:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   folklore
[14444:3632:0209/170103.661:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   family & parenting
[14444:3632:0209/170103.661:VERBOSE2:epsilon_greedy_bandit_model.cc(167)]   personal finance
[14444:3632:0209/170103.663:INFO:notification_helper_win.cc(171)] Focus Assist is disabled
[14444:3632:0209/170103.668:INFO:notification_helper_win.cc(208)] Notifications are enabled
[14444:3632:0209/170103.670:VERBOSE1:ad_notification_serving.cc(165)] Serve ad for segments:
[14444:3632:0209/170103.670:VERBOSE1:ad_notification_serving.cc(167)]   folklore
[14444:3632:0209/170103.677:VERBOSE1:ad_notification_serving.cc(167)]   family & parenting
[14444:3632:0209/170103.677:VERBOSE1:ad_notification_serving.cc(167)]   personal finance
[14444:3632:0209/170103.686:VERBOSE2:ad_notification_serving.cc(273)] 23 eligible ads before pacing
[14444:3632:0209/170103.686:VERBOSE2:eligible_ads_priority_filter.cc(81)] 23 eligible ads with a priority of 1
[14444:3632:0209/170103.691:VERBOSE2:ad_notification_serving.cc(278)] 23 eligible ads after pacing
[14444:3632:0209/170103.692:VERBOSE1:ad_notification_serving.cc(264)] Found 23 eligible ads
[14444:3632:0209/170103.695:VERBOSE1:ads_service_impl.cc(1769)] Timeout ad notification with uuid b8923d20-1d1b-4c56-9f94-58a11270d2c1 in 120 seconds
[14444:3632:0209/170103.696:VERBOSE1:ad_notification_serving.cc(101)] Ad notification delivered:
  uuid: b8923d20-1d1b-4c56-9f94-58a11270d2c1
  creativeInstanceId: e7706405-3ffb-48e9-8864-00aa3bf3dfbb
  creativeSetId: 6b869181-7330-4b9c-80b7-d33c50493b22
  campaignId: 37d14e05-fa1f-4730-8fd0-fa44a5ea41b3
  segment: folklore
  title: Gala Games: Revolution!
  body: Tired of the same repetitive gaming experiences? Join us!
  targetUrl: https://gala.games/gala-games-and-brave-present-mirandus-shops/
[14444:3632:0209/170103.704:VERBOSE3:ad_notification_event_viewed.cc(24)] Viewed ad notification with uuid b8923d20-1d1b-4c56-9f94-58a11270d2c1 and creative instance id e7706405-3ffb-48e9-8864-00aa3bf3dfbb
[14444:3632:0209/170103.707:VERBOSE1:confirmations.cc(90)] Confirming view ad for creative instance id e7706405-3ffb-48e9-8864-00aa3bf3dfbb

pulls value and value remained unchanged ( same as default value i.e pull 0 value 1.0) - logged epsilon_greedy_bandit_arms isn't displaying expected values for Brave Ads time out #14054

Case 3.7: Click two Ads of same segment

Ensured that the value for pull becomes 2 and value is 1.0

Case 3.8: Dismiss two Ads of same segment

Ensured that the value for pull becomes 2 and value is 0.0

srirambv · 2021-02-09T17:12:20Z

Verification passed on OnePlus 6T with Android 10 running 1.20.102 x64 build

Case 1: Default models enabled (Text Classification and Purchase Intent)

Used --args --enable-logging=stderr --vmodule="*/bat-native-ledger/*"=6,"*/brave_rewards/*"=6,"*/bat-native-ads/*"=6,"*/bat-native-confirmations/*"=6,"*/brave_ads/*"=6,"*/brave_user_model/*"=6 --brave-ads-staging --brave-ads-debug --rewards=staging=true,reconcile-interval=5

Verified Text Classification and Purchase Intent models are enabled and Epsilon Greedy Bandit is disabled
Triggered ad notification and verified that ads are served as usual via text classification
Verify that bandit is processing (despite the model being disabled, that is by design)

Verified an AD using the search keyword `coinbase earn` and ensured purchase intent for Keywords are working as expected

2021-02-09 16:28:40.793 20644-20644/? V/chromium: [VERBOSE1:purchase_intent_processor.cc(101)] Extracted purchase intent signal from visited URL
2021-02-09 16:28:40.870 20644-20644/? V/chromium: [VERBOSE1:text_classification_processor.cc(62)] Classified text with the top segment as sports-sports
2021-02-09 16:28:40.900 20644-20644/? V/chromium: [VERBOSE1:conversions.cc(151)] No conversions found for visited URL
2021-02-09 16:29:04.992 1305-1408/? I/OHPD: isIsolated process: app.uid:99495,app.info.uid:11227, app.processName:com.brave.browser:sandboxed_process1:org.chromium.content.app.SandboxedProcessService1:15
2021-02-09 16:29:04.993 1305-1408/? I/OHPD: isIsolated process: app.uid:99481,app.info.uid:11227, app.processName:com.brave.browser:sandboxed_process1:org.chromium.content.app.SandboxedProcessService1:8
2021-02-09 16:29:04.994 1305-1408/? I/OHPD: isIsolated process: app.uid:99479,app.info.uid:11227, app.processName:com.brave.browser:sandboxed_process1:org.chromium.content.app.SandboxedProcessService1:7
2021-02-09 16:29:05.689 1305-1408/? I/OHPD: isIsolated process: app.uid:99495,app.info.uid:11227, app.processName:com.brave.browser:sandboxed_process1:org.chromium.content.app.SandboxedProcessService1:15
2021-02-09 16:29:05.690 1305-1408/? I/OHPD: isIsolated process: app.uid:99481,app.info.uid:11227, app.processName:com.brave.browser:sandboxed_process1:org.chromium.content.app.SandboxedProcessService1:8
2021-02-09 16:29:05.691 1305-1408/? I/OHPD: isIsolated process: app.uid:99479,app.info.uid:11227, app.processName:com.brave.browser:sandboxed_process1:org.chromium.content.app.SandboxedProcessService1:7
2021-02-09 16:29:17.872 20644-20644/? V/chromium: [VERBOSE1:subdivision_targeting.cc(168)] Fetch subdivision target
2021-02-09 16:29:17.872 20644-20644/? V/chromium: [VERBOSE2:subdivision_targeting.cc(169)] GET /v1/getstate
2021-02-09 16:29:17.873 20644-20644/? V/chromium: [VERBOSE5:subdivision_targeting.cc(173)] URL Request:
      URL: https://ads-serve.bravesoftware.com/v1/getstate
      Method: BraveAdsUrlRequestMethod::GET
2021-02-09 16:29:17.910 20644-20644/? V/chromium: [VERBOSE1:subdivision_targeting.cc(189)] Successfully fetched subdivision target
2021-02-09 16:29:17.920 20644-20644/? V/chromium: [VERBOSE1:subdivision_targeting.cc(267)] Fetch ads subdivision target in 0 hours, 3 minutes, 54 seconds at 4:33 PM
2021-02-09 16:29:26.019 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(165)] Serve ad for segments:
2021-02-09 16:29:26.019 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(167)]   sports-sports
2021-02-09 16:29:26.019 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(167)]   hobbies & interests-coins
2021-02-09 16:29:26.019 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(167)]   education-education
2021-02-09 16:29:26.019 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(167)]   Crypto Purchase Intent
2021-02-09 16:29:26.023 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(182)] No eligible ads found for segments
2021-02-09 16:29:26.023 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(197)] Serve ad for parent segments:
2021-02-09 16:29:26.023 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(199)]   sports
2021-02-09 16:29:26.023 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(199)]   hobbies & interests
2021-02-09 16:29:26.024 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(199)]   education
2021-02-09 16:29:26.024 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(199)]   Crypto Purchase Intent
2021-02-09 16:29:26.028 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(214)] No eligible ads found for parent segments
2021-02-09 16:29:26.028 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(226)] Serve untargeted ad
2021-02-09 16:29:26.033 20644-20644/? V/chromium: [VERBOSE1:eligible_ad_notifications.cc(70)] All advertisers have been shown, so round robin
2021-02-09 16:29:26.034 20644-20644/? V/chromium: [VERBOSE1:client.cc(385)] Resetting seen advertisers
2021-02-09 16:29:26.042 20644-20644/? V/chromium: [VERBOSE2:ad_notification_serving.cc(273)] 2 eligible ads before pacing
2021-02-09 16:29:26.042 20644-20644/? V/chromium: [VERBOSE2:eligible_ads_priority_filter.cc(81)] 2 eligible ads with a priority of 1
2021-02-09 16:29:26.043 20644-20644/? V/chromium: [VERBOSE2:ad_notification_serving.cc(278)] 2 eligible ads after pacing
2021-02-09 16:29:26.043 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(264)] Found 2 eligible ads
2021-02-09 16:29:26.065 20644-20644/? V/chromium: [VERBOSE1:ad_notification_serving.cc(101)] Ad notification delivered:
      uuid: c6d6cceb-c645-4e6e-a522-b6eb017d55da
      creativeInstanceId: 30ecd70f-f56d-4745-a457-4035fcab54b6
      creativeSetId: 0a697a3b-233a-44bc-a337-e2441fb277a7
      campaignId: d4d0474c-277a-4714-b25d-7b484cdb580a
      segment: untargeted
      title: This is a test
      body: This is a test
      targetUrl: https://www.brave.com
2021-02-09 16:29:26.085 20644-20644/? V/chromium: [VERBOSE3:ad_notification_event_viewed.cc(24)] Viewed ad notification with uuid c6d6cceb-c645-4e6e-a522-b6eb017d55da and creative instance id 30ecd70f-f56d-4745-a457-4035fcab54b6
2021-02-09 16:29:26.086 20644-20644/? V/chromium: [VERBOSE1:confirmations.cc(90)] Confirming view ad for creative instance id 30ecd70f-f56d-4745-a457-4035fcab54b6

Verified model is not suggesting segments to the ad serve pipeline and didn't see Exploring epsilon greedy bandit segments: ... or Exploiting epsilon greedy bandit segments: ...

Case 2: Variant models enabled

Verified that Purchase Intent models and Epsilon Greedy Bandit are enabled and Text Classification is disabled when the brave is launched using cmd line: --enable-logging=stderr --vmodule="*/bat-native-ads/*"=9,"*/brave_ads/*"=9,"*/brave_user_model/*"=9,"*/bat_ads/*"=9 --brave-ads-staging --rewards=staging=true --disable-features="TextClassification" --enable-features="EpsilonGreedyBandit"
Viewed an AD using the search keyword best bitcoin lending sites and ensured purchase intent for Keywords are working as expected
Verified that bandit is processing
Verified model is suggesting segments to the ad serve pipeline and didn't see Exploring epsilon greedy bandit segments: ... or Exploiting epsilon greedy bandit segments: ...

moritzhaller assigned moritzhaller and tmancey Jan 6, 2021

moritzhaller added OS/Desktop enhancement feature/ads priority/P3 The next thing for us to work on. It'll ride the trains. QA/Yes labels Jan 6, 2021

tmancey mentioned this issue Jan 7, 2021

Add eligible segment filter for bandits brave/brave-core#7533

Merged

23 tasks

moritzhaller closed this as completed in brave/brave-core#7533 Jan 7, 2021

tmancey added this to the 1.20.x - Nightly milestone Jan 8, 2021

moritzhaller added the release-notes/exclude label Jan 9, 2021

kjozwiak added the OS/Android Fixes related to Android browser functionality label Jan 26, 2021

stephendonner mentioned this issue Jan 29, 2021

Text classifier runs even when explicitly disabled via command-line #13840

Closed

stephendonner added the QA Pass-macOS label Feb 4, 2021

btlechowski added the QA Pass-Linux label Feb 8, 2021

GeetaSarvadnya mentioned this issue Feb 9, 2021

epsilon_greedy_bandit_arms isn't displaying expected values for Brave Ads time out #14054

Closed

GeetaSarvadnya added the QA Pass-Win64 label Feb 9, 2021

srirambv added the QA Pass - Android ARM label Feb 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add eligible segment filter for bandits #13395

Add eligible segment filter for bandits #13395

moritzhaller commented Jan 6, 2021

kjozwiak commented Jan 26, 2021

stephendonner commented Feb 4, 2021 •

edited by btlechowski

Loading

GeetaSarvadnya commented Feb 8, 2021 •

edited

Loading

srirambv commented Feb 9, 2021

Add eligible segment filter for bandits #13395

Add eligible segment filter for bandits #13395

Comments

moritzhaller commented Jan 6, 2021

kjozwiak commented Jan 26, 2021

stephendonner commented Feb 4, 2021 • edited by btlechowski Loading

GeetaSarvadnya commented Feb 8, 2021 • edited Loading

srirambv commented Feb 9, 2021

stephendonner commented Feb 4, 2021 •

edited by btlechowski

Loading

GeetaSarvadnya commented Feb 8, 2021 •

edited

Loading