Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sprint - Nov 11 to Nov 22 #26021

Closed
marandaneto opened this issue Nov 6, 2024 · 13 comments
Closed

Sprint - Nov 11 to Nov 22 #26021

marandaneto opened this issue Nov 6, 2024 · 13 comments
Labels
sprint Sprint planning

Comments

@marandaneto
Copy link
Member

Global Sprint Planning

3 things that might take us down

Urgent incident follow-ups

https://github.com/orgs/PostHog/projects/103/views/2

Team sprint planning

For your team sprint planning copy this template into a comment below for each team.

# Team ___

**Support hero:** ___

## Retro

<!-- Grab the high and low priority items from last time and add whether that item was completed or not -->

- 

## Hang over items from previous sprint

<!-- For each item, decide to re-prioritise (and add below) or deprioritise -->

- Item 1. prioritised/deprioritise

## OKR

1. OKR, status (red/yellow/green) and action points if yellow/red


### High priority

-

### Low priority / side quests

-
@marandaneto marandaneto added the sprint Sprint planning label Nov 6, 2024
@marandaneto marandaneto pinned this issue Nov 6, 2024
@mariusandra
Copy link
Collaborator

mariusandra commented Nov 6, 2024 •
edited by benjackwhite
Loading

Team Can Direct Priorities

  • Support: Ben

Retro

  • @mariusandra Get messaging v0 ready for dogfooding.
    • ⚪ out for testing, but see below (on hold)
  • @benjackwhite CDP UI/UX + coming back from holiday (odds/ends)
    • 🔴 catching up took time, no work on CDP, that's for next week
  • @oliverb123 Olly 80% on error tracking. Improve live debugging of CDP.
    • 🟢 merged PR for better error handling
    • 🟢 error tracking --> being built
  • @MarconLP Support sprint + new destinations
    • 🟢 coming along, being built

OKRs

  • Widespread usage of the new CDP
  • Messaging V1
  • Ingestion is rock solid

Sprint focus

High priority

  • CDP focus on ad conversion platforms integrations

@pauldambra
Copy link
Member

pauldambra commented Nov 6, 2024 •
edited by marandaneto
Loading

Team Claire will leave me if I travel in December as well

most in the whole sprint

Support hero: @marandaneto

Retro

  • yiannis - Autocapture has been at “almost ready” for too long. 98% of the work was done early and fast. Then some minor refactoring, some slow feedback loops and some indecision from my part for minor details. Also didn’t take documentation into account
  • yiannis - Still in a “need-to-ask-permission” mindset that I meed to grow out of. Working on this :)
  • yiannis - Been getting more comfortable and faster with community responses and support
  • paul - support feels pretty draining at the moment. i went through a bunch of tickets and questions / setup advice is a big chunk of load, will go back through those specifically and see if there's anything we can fix here
  • manoel - medical leave after surgery and offsite, just worked on fixing bugs the last and this sprint

Mobile

Replay

  • replay URL denylist @richard-better
    • also interrupted by travel
    • feature is working and we're making sure it's releasable
  • a brief diversion before querying network data - wrap websockets as well as fetch @pauldambra
    • lots of user support meant i made very little progress on this 😵

Low priority / side quests

  • ✅ nobody is lost or arrested in Valencia
  • 🔄 trigger to start recordings on event @richard-better
  • 🔄 lazy loading replay in the SDK @pauldambra

OKR

  1. OKR, status (red/yellow/green) and action points if yellow/red

🟡📱Goal 1: People think of PostHog as a mobile solution
🟡 📹 Goal 2: Replay playback 3000
- delayed-ish till Nov and Cory time in Valencia
🟡 🎤 Goal3: Ship the things users keep asking for

High priority

Mobile

Replay

  • a brief diversion before querying network data - wrap websockets as well as fetch @pauldambra
  • finish replay URL denylist and then trigger replay on event @richard-better

Low priority / side quests

@Phanatic
Copy link
Contributor

Phanatic commented Nov 6, 2024

Team Surveys

Retro

✅ Beta testing no-code experiments with customers
✅ Surveys bug fixes
✅ No-code experiemnts bug fixes
✅ Investigate adaptive sampling for collecting survey responses.
✅ Support queues were split
⌛ Feature preview release of no-code experiments
⌛ Documentation for no-code experiments.

Side Quests

OKRs

  1. Implement Enterprise-ready features for Surveys
  2. Increase adoption of Surveys.

High priority

  • Adaptive sampling for collecting survey responses.
  • Support no-code experiments.

@jurajmajerik
Copy link
Contributor

jurajmajerik commented Nov 6, 2024 •
edited
Loading

Team Experiments

Retro

  • 🟢 @danielbachhuber joined the team! 🥳

    • @raquelmsmith is onboarding him in person
    • First PRs in: UI improvements to Experiment goal, onboarding flow, up to 10 secondary metrics
    • Running first experiment - select multiple products in the product screen

  • 🟢 [priority_customer]

  • 🟡 Continued on HogQL for Experiments @jurajmajerik (Experiments HogQL rewrite #24914)

    • Opted for a large refactor of the Experiments logic
    • Shared components (selectors) with Product Analytics, but tightly coupled with their logic. Convoluted state flows, hard to reason about.
    • Forked these components - we want them to work with our logic
    • We want to be in control of these components - make sure the statistics always work correctly for any goal metrics.
    • This is taking extra time/delaying the HogQL completion, but worthwhile, will enable us to move faster.

  • 🟢 Review statistics with a contractor @jurajmajerik

    • Latest feedback incorporated, seem to be done for now - all main metric types supported.
    • Next step: plug these methods into our APIs and write tests
    • Might need some tweak later

  • 🟢 Support hero

    • Experiments queue is set up now
    • Workload manageable

Sprint goals

  • Complete the onboarding list (mostly UI improvements) @danielbachhuber
  • Data Warehouse integration for [priority-customer] - pending information from them - start collecting requirements @danielbachhuber
  • HogQL rewrite @jurajmajerik
    • Finish Experiments logic refactor
    • Migrate Experiment records
    • Test behind feature flag
  • Support hero @jurajmajerik

@robbie-c
Copy link
Member

robbie-c commented Nov 6, 2024 •
edited
Loading

Team web analytics

Support hero: @robbie-c

Retro

Onboarding Mike, currently doing a hackathon. Cookieless prototype is pretty close but dealing with a couple of edge cases.

Tasks

🟠 Finish cookieless prototype
🟢 Onboard Mike / have a successful offsite
🟢 Finish up the v1 sessions table PR

OKR

  • Continue to improve performance
  • Build a small amount of more advanced marketing features, to measure interest
  • Cookieless tracking
  • Improve the NextJS experience
  • More page-level insights
  • Improve the documentation, and link to it more from the app

High priority

Mike

Robbie

Stretch goals

Mike

@fuziontech
Copy link
Member

fuziontech commented Nov 6, 2024 •
edited
Loading

Team 🧊 Data

OKR Q2 2024

Objective

Goals for Q4:

  • Improve elasticity and flexibility of our data store by putting all our data in Iceberg
    • Work with Altinity effectively to ship read path for Iceberg on ClickHouse - Brett Hoerner
    • Setup infrastructure to ship all of our data to Iceberg on S3 - James Greenhill
    • Shipping Query logs to Iceberg - James Greenhill
  • Continue improving CH operational expertise
    • Upgrade to a later version of Clickhouse - James Greenhill
    • Capacity planning - James Greenhill
    • Automation - Daniel Escribano
    • Put some of the basic mitigation operations in runbooks - Daniel Escribano
  • Schema management
    • Tool for schema migration (coordinator schemas) - Daniel Escribano
    • Tool for long running mutations - Daniel Escribano
  • Continued investment in performance
    • Tooling for other teams to understand which queries are slow and why - Ted Kaemming
    • Investigate variability of queries - Ted Kaemming
    • Per-team limits on queries/query complexity (needs product work) - Ted Kaemming

Board

https://github.com/orgs/PostHog/projects/85/views/2

Retro

🔥 Incidents 🔥

Last Sprint's Goals

📣 Most work has been re-prioritized mid week in order to support snapshot diffing and re-clustering US
Tasks impacted marked with ❌ ♻️

This Sprint's Goals

TL;DR

@EDsCODE
Copy link
Member

EDsCODE commented Nov 6, 2024

Team Data Storehouse

OKR Q4 2024

Objective

Query 3000

  • Key Results:
    • Autocomplete
    • Increase general BI experience/product BI meta#157
    • Declutter the data warehouse UI and make the features intuitive to find

Data Modeling MVP

  • Key Results:
    • Infrastructure decided and implemented
    • Integrating external data with feature flags
    • External data everywhere in insights/persons/cohorts
    • Get billing team to use modeling in posthog for their invoices_with_annual table

Retro

  • Using warehouse person properties in experiment metrics
  • SQL editor @EDsCODE
  • release dashboard insight variables to everyone @Gilbert09
  • BigQuery source release to everyone @Gilbert09
  • Investigate reducing temporal-data-warehouse pod size @Gilbert09
  • Resolving batch exports replication lag - some solution, working with team clickhouse @tomasfarias
  • More onboarding tasks @rossgray
    • Release chargebee source
    • Select series colors in BI
    • HogQL ambiguous query error when joining tables with the same field names
    • BI breakdowns

Priority

  • Release dashboard insight variables @Gilbert09
  • BigQuery source release to everyone @Gilbert09
  • Reduce temporal-data-warehouse pod size (memory) @Gilbert09
  • SQL editor 3000 MVP @EDsCODE
  • SPMC Batch Exports @tomasfarias
  • Splitting persons batch exports @tomasfarias
  • HogQL ambiguous query error when joining tables with the same field names @rossgray
  • One of: Backfill UI, test connection button, or sessions/groups batch exports @rossgray

@daibhin
Copy link
Contributor

daibhin commented Nov 6, 2024 •
edited
Loading

Team Offsite Gate Crashers

@neilkakkar working on Experiments for 1 more week (apparently)

Retro

🟢 Stack trace demangling enabled for PostHog team
More work than expected here and lost time to modelling churn
Strong progress at offsite. End-to-end resolution, in-memory caching and saving for frames & source maps
Still need to put all the pieces together and deploy to test with live events

🔴 In-app source map uploading @daibhin
Lost some time to the rrweb upgrade
Good progress but has dependency on demangling goal. Carried over

✅ Frontend query caching to improve page load speed @daibhin

OKR

🟢 Human readable stack traces for Javascript
🟢 Ensure scalability of query time issue grouping
⚪️ Alerting via CDP
🟢 Launch

High priority

Deploy cymbal end-to-end @oliverb123

  • Configure S3 env variables for deployment
  • Writing events to ClickHouse in cymbal rather than plugin server (for team 2)
  • Expected long tail of edge case handling / papercuts
    • Exceptions without a stack trace
    • Liveness checks
    • Evaluation of scaling / perf

In-app stack trace @daibhin

  • Upload source maps on unresolved frames
  • Redesign stack trace display component
  • Investigate adding adjacent lines as context to an individual frame

Investigate build time source map uploading @neilkakkar (?)

@zlwaterfield
Copy link
Contributor

zlwaterfield commented Nov 6, 2024 •
edited
Loading

Team Growth

Retro

  • @zlwaterfield
    • RBAC
      • Finish RFC for billing and get feedback on it
      • Start working on it - get at least 1 PR merged for it
    • Get the in-app credit card entry experiment shipped
    • Make a few improvements to the technical exercise (based on using it for 2 weeks)
    • extra
      • picked up teams trials (live)
      • billing on call
      • credits eligibility lowered
      • sso enforced invite team member flow
  • @raquelmsmith
    • Slowly roll out personless default change, deal with any issues - holding off for now
    • Fix bugs with product analytics templates onboarding flow
    • Ship session replay "templates" screen
    • Write data warehouse activation query
    • Think about how to redo onboarding so people can pick multiple products at once
    • Respond to customer about projects-on-invites
  • @patricio-posthog
    • Billing limit: Get this bug fixed once and for all so we can move on
    • Forecasting bug: Fix forecasting and if it's a good idea, reimplement it with a new version
    • Improve current billing API endpoints (optimize or spit current ones into separate endpoints)
    • Check if the webhooks for invoices are working fine (check events at Posthog) / turn off the daily sync to test that.
    • extra
      • admin improvements for viewing json and permissions
      • running usage reports via admin (no more sshing needed)

Q4 Goals

✅=finished 🟡=in progress 🔴=won't finish ⚪=not started

  1. 🟡 Get to a 4/10 in Enterprise platform support (Zach)
    • 🟡 Release Enterprise addon and iterate with sales to find a pricing model that makes sense
    • 🟡 Build RBAC
  2. 🟡 Make revenue & forecasting more reliable (Patricio)
    • 🟡 Rejig how we do sync_invoices so we don’t see big drops - respond to webhooks instead of large syncs
    • ⚪ Evaluate temporal vs celery for usage reports and billing tasks so we can adhere to rate limits better
    • ⚪ Side hustle, if time permits: Rejig how anonymous / identified events are billed, as the current implementation is confusing.
  3. 🟡 Activation: reverse trials, analytics onboarding flow, understand CC activation (Raquel)
    • ⚪ Dig into the data, watch replays to figure out why CC activation rate has been falling
    • ⚪ Add reverse trial to onboarding
    • ✅ Add reverse trial to billing page for teams

This sprint

Time off: Zach(1/2 day), Raquel (1 day)

  • @zlwaterfield
    • RBAC
      • Get access models shipped
      • Get base middleware implements
      • Pick first product and get the base RBAC setup for it
    • Get Teams Trials fully live
    • Ship lowering of self serve credits requirements
    • Finish and ship project limits API checks
    • Onboarding for Surbhi
    • if time:
      • (from last time) Make a few improvements to the technical exercise (based on using it for 2 weeks)
      • (from last time) Get the in-app credit card entry experiment shipped
      • Review existing billing metrics, hookup to billing on call, see if we should add others
      • customer io transaction api
      • More billing metrics for on call
  • @patricio-posthog
    • Continue with forecasting bugs, trying to understand what's happening and how to solve it.
    • (From last time): Turn off daily invoice sync to rely 100% on webhooks.
    • Try to optimize /billing endpoint, maybe separating forecasting into its own endpoint (after fixing it and understand how it works)
      Extra things if there is time:
    • Start to analyze forecasting and see its accuracy after each period.
  • @raquelmsmith
    • Get product analytics dashboard templates out again
    • follow up on replay templates / reengagement campaign
    • Daniel in-person onboarding
    • Multi-product-select in onboarding (with Daniel)

@aspicer
Copy link
Contributor

aspicer commented Nov 6, 2024 •
edited
Loading

Team Product Analytics

Support hero: @aspicer and @anirudhpillai 

Q4 2024 objectives

Legacy 0x - Thomas owning

  • All insights are query-based
  • rm -rf posthog/queries/
  • Insight query is stored in the URL for instant sharing of unsaved insights

First-class environments - Michael owning

  • Environments are rolled out to everyone

Threshold-based alerts - Anirudh owning

  • All users get to set up alerts on PostHog metrics
  • Alerts are launched with Marketing, and positioned into paid plans
  • We're using alerts ourselves for key trends and funnels

Flexible funnels - Sandy owning

  • UDF funnels rolled out to everyone, unlocking arbitrary complexity
  • Allowing parts of funnels to be unordered. No more separation between regular, unordered, and strict funnels
  • Optional funnel steps

10x onboarding and ease of discovering features - Anirudh and Sandy owning

  • We run an exercise in implementing PostHog in a hobby project. What stands out as painful or unclear?

User delight 2x - support rotation owning

  • Support is 1 hero + 1 sidekick
  • Every week support folks ship those small requested features (see list of requests so far)
  • Goal: 90% of tickets fulfill the SLA

B2B analytics research - Anna owning

  • Understand the market and user needs better in order to improve our current Group analytics product next quarter

Retro

Sandy

  • ✅ Support (5 days)
  • 🟢 24.8 query porting
  • 🟡 Continue porting cohorts
  • ✅ Funnel exclusion fixes

Thomas (Returns on Nov 4th!)

Anirudh (OOO through Nov 6th)

  • 🟢 Resume alerting roll-out

Upcoming Sprint

Anirudh (Support Nov 11 - 15)

Thomas

Sandy (Support Nov 18 - 22)

@Twixes
Copy link
Member

Twixes commented Nov 6, 2024 •
edited
Loading

Team AI Product Manager

Retro

High priority

  • 🟢 Funnels support @skoob13
  • 🟢🔁 Tune agent/prompt for more relevant results (primary: excluding most $-prefixed events/properties) @skoob13 @Twixes
  • ➡️ Stream feedback on the agent's current task in conversation @Twixes
  • ➡️ PostHog dashboard for AI product manager use @Twixes
  • ➡️ Generalize Session Replay's OpenAI data processing opt-in (to be determined with @fraserhopper) @Twixes

Low priority / side quests

  • ➡️ Results evaluation framework - a benchmark @skoob13
  • 🟡 Environments out as a feature preview @Twixes

Hang over items from previous sprint

  • Generalize Session Replay's OpenAI data processing opt-in (to be determined with @fraserhopper)
  • Stream feedback on the agent's current task in conversation
  • PostHog dashboard for AI product manager use
  • Results evaluation framework - a benchmark

Objective

LLM-based product manager as a service

High priority

  • Results summarization
  • Generalize Session Replay's OpenAI data processing opt-in (to be determined with @fraserhopper) @Twixes
  • Stream feedback on the agent's current task in conversation @Twixes
  • PostHog dashboard for AI product manager use @Twixes
  • Tune Trends generation for more relevant results (smarter application of breakdowns, better property value examples) @skoob13
  • Handle errors (agent recursion limit + graceful handling of e.g. ClickHouse errors) @skoob13

Low priority / side quests

  • Results evaluation framework - a benchmark @skoob13
  • Environments out as a feature preview @Twixes

@benjackwhite
Copy link
Contributor

Team Infrastructure

Support hero: @danielxnj

Retro

  • 🟡 Finish ArgoCD migration @bciaraldi
  • 🟢 Finish spot instances (at least for Decide and events, which are most expensive and can handle random load the best) @xneyder
    • Decide/events/monitoring all moved over
    • Tuned a bunch of limits and instance types to match a spot workload better
    • Final more sensitive services like recordings and ingestion need some more tweaking
  • 🔴 Reserve instances @xneyder
    • Purposefully delayed until spot work is fully done as we are making big savings there already that affect the calculation
  • 🔴 Move most web/workers to same AZ as Clickhouse + easy failover.
    • (Will follow up after)
  • (Half the time taken up by onboarding/offsite)

OKR

  1. 🏃 Continuous Delivery
  2. 💰 Cost visualization
  3. 🧪 Continuous Integration

High priority

  • 100% do the reservation changes ASAP - @benjackwhite
  • Fine tune our instance types to match our deployment workloads (cost optimisation to utilise the most of a node) @danielxnj
  • Optimize the LB/envoy deployments to remove all cross AZ traffic (cost optimization) @MichaelKutsch-ph
  • Tune the Warpstream dedicated node pools @danielxnj
  • ArgoCD - move lowest priority posthog services over (cdp, temporal-workers, cymbal) @bciaraldi
  • (Will potentially update some outcomes of the offsite later)

Low priority / side quests

  • Docker cache (working with Tom as warehouse still uses minio) @danielxnj

@dmarticus
Copy link
Contributor

dmarticus commented Nov 6, 2024 •
edited
Loading

Team Flags

Support

Now that we've split the queues, I'm the permanent support person for all things, but as Haven ramps up on support we will start to do a support hero rotation between the two of us. Will have an update on that timeline by next sprint.

Retro

Seeing some non-deterministic behavior from the maxmindDB; seems like it starts to return different values from the IP address we expect sometimes. Still figuring out exactly what the resolution is here (see thread: https://posthog.slack.com/archives/C06GG249PR6/p1730842770193459), but wanted to raise here since it's a cross-team dependency.

Things we got done

  • Finish static and dynamic cohort matching for new feature flag service
    • support full subset of cohort match operations (not_in, etc) at the service/DB level
  • Prepare for Haven's onboarding in Seattle

Things we moved over

  • Port over existing feature flag integration tests from the Django to rust repo (lots of valuable alpha in our existing /decide tests, but this doesn't need to happen this sprint) - @dmarticus

OKRs

  1. Ship new feature flags in production and accomplish the 10x scale + 2x speed boost performance targets (ideally decreases infra spend by N% as well, will try to figure out a more specific target going forward)
  2. Maintain current /decide service and make optimizations.

Sprint

High priority

(all new flags work is tracked in this mega-ticket: #22131)

@Twixes Twixes unpinned this issue Nov 25, 2024
@Twixes Twixes closed this as completed Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
sprint Sprint planning
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests