ProjectProposal_Group011_WI24.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# COGS 108 - Project Proposal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Names\n",
    "\n",
    "- Sheena Patel\n",
    "- Anna Chen\n",
    "- Shreya Velagala\n",
    "- Catherine Du\n",
    "- Esther Liu\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Research Question"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b> What is the relationship between time and interest in various TikTok fashion trends? </b> <br>\n",
    "\n",
    "In Depth RQ: <br>\n",
    "How can we predict the level of interest in various TikTok fashion trends across New York, California, and Texas using TikTok trend interest score data from January 2019 to December 2023 using a model that inputs the fashion trend name, monthly time data, and region to predict an interest rating between 0 - 100 and an associated label of 'low' (interest score < 25), 'rising' (interest score < 50), 'popular' (interest socre < 75), and 'trending (interest score > 75) for the specified input? <br>\n",
    "\n",
    "This question aims to develop a predictive model that evaluates the popularity of TikTok fashion trends in different regions and times, using a quantifiable interest rating system. The focus on a state-by-state analysis allows for a detailed understanding of regional preferences and trends overtime. <br>\n",
    "\n",
    "Interest Score/Interest Over Time Definition:  <br>\n",
    "The \"interest score\" on Google Trends represents the relative popularity of a search query in a\n",
    "specific region and time frame. It is indexed from 0 to 100, where 100 signifies the peak popularity\n",
    "for the term. This score does not indicate the absolute search volume but rather shows the search term's popularity relative to the highest point on the chart for the given region and time. A higher score means more people are searching for that particular term at that time, while a lower score indicates lesser interest. The data is useful for identifying trends and understanding how interest in certain topics changes over time. <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Background and Prior Work"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our team is curious to understand how current fashion trends can be understood to predict the rating of a piece of clothing. We aim to create a prediction model that can take types of clothing and dates as input to predict the rating of a piece of clothing. We hope this model will have real-world application by possibly allowing companies to predict ratings for different clothing pieces based on GenZ fashion to potentially improve clothing purchase rates. Many datasets exist that contain information about women’s clothing including product descriptions, reviews, and ratings. However, we would like to incorporate TikTok trends into our dataset to use TikTok fashion trends as a data feature when predicting the rating for clothing. \n",
    "\n",
    "Currently, we have a few datasets including Amazon Women's Fashion Dataset<a name=\"cite_ref-1\"></a>[<sup>1</sup>](#cite_note-1) and Women's Ecommerce Clothing Reviews<a name=\"cite_ref-2\"></a>[<sup>2</sup>](#cite_note-2) that seem relevant to our project goals. Based on our initial research, our ideal dataset would include columns for clothing type, clothing description, clothing review, clothing rating, and a label for each row representing what type of TikTok trend the clothing item falls into based on the clothing description. Additionally, we found other projects that have approached similar problems. One example is this project: Amazon Women's Clothing Review<a name=\"cite_ref-1\"></a>[<sup>1</sup>](#cite_note-1). In the project, the researcher performs EDA by analyzing various trends in the sentiment for Amazon clothing reviews. We plan to incorporate a similar approach to our EDA by visualizing the trends in rating prediction, but analyzing prediction from the perspective of TikTok trends and views for different TikTok trends influencing rating. Another project that had a similar approach is Rating prediction<a name=\"cite_ref-3\"></a>[<sup>3</sup>](#cite_note-3). This project performs rating prediction similar to what we would like to do. My team plans to use this project as an example when creating our prediction model and also use a similar approach for EDA using TF-IDF to assign TikTok trends to different clothing descriptions. \n",
    "\n",
    "The first project<a name=\"cite_ref-3\"></a>[<sup>3</sup>](#cite_note-3) found that the main trigrams from the reviews fall into the positive sentiment categories. For instance, the research found that fit true size, run true size, fit just right, love love love, fit like glove, usual wear size, and every time wear were some of the most prominent trigrams. This research conducts more of a sentiment analysis to show that reviews and ratings are more positive in their dataset. The second project<a name=\"cite_ref-1\"></a>[<sup>1</sup>](#cite_note-1) performed EDA and tried to analyze what kinds of age groups make what kinds of reviews and ratings. The research project found that layering, lounge and swim clothing tend to have the best reviews. The research also interestingly found that higher age groups had worse ratings. What we found interesting about this project is that TF-IDF Vectorization was used to convert the clothing descriptions into vectors to be passed as input to the prediction model. I think we are going to have to take a similar approach to our project.\n",
    "\n",
    "1. <a name=\"cite_note-1\"></a> [^](#cite_ref-1) Jaewook. (2022, October 13). Amazon reviews on Women Dresses. Kaggle. https://www.kaggle.com/code/jaewook704/amazon-reviews-on-women-dresses\n",
    "2. <a name=\"cite_note-2\"></a> [^](#cite_ref-2) Agarap, Abien Fred, and Nicapotato. (2018, Feburary 3). Women’s e-Commerce Clothing Reviews. Kaggle. https://www.kaggle.com/datasets/nicapotato/womens-ecommerce-clothing-reviews\n",
    "3. <a name=\"cite_note-3\"></a> [^](#cite_ref-3) Matteoanzano. (2023, December 5). Customer review analysis with text mining. Kaggle. https://www.kaggle.com/code/matteoanzano111/customer-review-analysis-with-text-mining/input"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hypothesis\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "These are a few possible hypotheses under exploration:\n",
    "- Clothing featured on the top trends of TikTok are most likely to have or drive a higher rating/sales rate.\n",
    "- More skirts will be sold at the peak of TikTok trends when TikTok celebrities wear them. \n",
    "- Clothing that isn’t trending has low ratings \n",
    "- Bright colors are more likely to be sold/rated higher in the summer than in other seasons. \n",
    "- More knit material in winter and cotton material in summer sold compared to other seasons of the year. \n",
    "- More blouses in summer and sweaters in winter sold compared to other seasons of the year. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our goal is to boost clothing sales for retail companies by considering recent fast fashion trends. This could help us provide companies with insight on whether a product is likely to sell or not. We want to merge a given ratings dataset of recent clothing styles and their respective ratings with a recent TikTok trends dataset and see if the clothing recommendations trending on tiktok have impacted sales/ ratings in the past so companies can base their clothing drops on recent trends. \n",
    "\n",
    "Ideally, we could find a tiktok dataset filtered to “fashion and clothing Tok” that has a column of words commonly mentioned in recent tiktok such as “pink”, “low-waisted”, “corset” and a column on the right that provides the amount of time this word was mentioned in the recent 4 months or 120 days. This word data could be scraped from speech to text transcription caption overlay, image recognition, comments, or hashtags on TikTok.\n",
    "\n",
    "What variables would you have? How would they be stored?\n",
    "- Clothing Type (Str)\n",
    "  - Jean, shirt, jacket, coverup, etc.\n",
    "- Customer Age (Int)\n",
    "  - numerical\n",
    "- Clothing features (Str)\n",
    "  - Buttons \n",
    "  - Zipper\n",
    "  - Slit \n",
    "- Clothing Style (Str)\n",
    "  - Streetwear, casual, fancy, work, formal, summer\n",
    "- Clothing Color (Str)\n",
    "  - Neutral color\n",
    "  - bright color\n",
    "- Tiktok Features (Int)\n",
    "  - Numerical Scale based on # times words from features appear in top TikTok hashtags\n",
    "- Date Purchased OR Data Rated (Time Series)\n",
    "  - Season \n",
    "  - Holiday\n",
    "- Rating (Int)\n",
    "  - 1-5 scale\n",
    "- Review Text  (Str)\n",
    "- Cost (Float)\n",
    "  - US Dollars\n",
    "\n",
    "How many observations would you have?\n",
    "- We would have about 10,000+ observations or 10,000 different clothing purchases in order to get an accurate model and avoid bias. \n",
    "\n",
    "What/who would the observations be? Over what time period? Etc.\n",
    "- Each observation would be an clothing purchase. We would aim for a period in the past 5 years (anywhere from 2019-2024) to get recent and up-to-date data. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ethics & Privacy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Are there any biases/privacy/terms of use issues with the data you propsed?\n",
    "- Accessing TikTok's API is crucial for our analysis, as it provides access to public-facing data, such as like/comment counts and trending hashtags, which are already readily available on a typical TikTok user's account. Moreover, our dataset will focus on sales information while safeguarding the anonymity of both customers and stores involved. Preserving this anonymity is essential to uphold privacy standards and ensure the ethical handling of data.\n",
    "\n",
    "Are there potential biases in your dataset(s), in terms of who it composes, and how it was collected, that may be problematic in terms of it allowing for equitable analysis? (For example, does your data exclude particular populations, or is it likely to reflect particular human biases in a way that could be a problem?) How will you set out to detect these specific biases before, during, and after/when communicating your analysis?\n",
    "- The United States boasts the largest share of TikTok users globally. As we analyze data patterns from TikTok and potentially from major American retail chains, we will exercise caution in extrapolating our findings to other countries. It's important to recognize that our dataset predominantly represents users who actively engage with TikTok and have consistent internet access. Consequently, we will examine factors contributing to the exclusion of certain demographics or stores from our analysis and assess the potential impact on our results. While acknowledging the influence of additional variables such as zoning and access to clothing stores on sales, we hope that our data could provide valuable suggestions for companies aiming to align their clothing offerings with popular social media trends.\n",
    "\n",
    "How will you handle issues you identified?\n",
    "- In order to handle issues related to biases and privacy in our datasets, we will emphasize that the trends we seek to predict and their impacts apply primarily to the states where the majority of this data is collected. We can’t seek to predict trends worldwide with the current biases of including majority American brands and a tik tok audience with a majority demographic from the states. Additionally, it may be hard to address issues of racial representation in the dataset, or biases towards specific bodies in relation to clothes trends. Although these issues may arise, our best course of action is to research and incorporate ethically sourced data. Our focus on choosing quality datasets from the start will ensure our analysis can be ethically predictive of the population.\n",
    "- Privacy can be maintained by not including any identifying information of users in our data analysis. When analyzing the datasets, the only relevant information would be what is already available to the public, such as when a user comments. Beyond that, identifying information such as, user handle, follower count will not be included. We will only include what is essential and we will be extra careful when determining what information we need. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Team Expectations "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Regular meetings/checkups every Tuesday at 7pm\n",
    "* Use discord and messages as main form of communication\n",
    "* If issues with communication arises, attempt to solve them as a team\n",
    "  * If a team member is non-responsive after that, contact the professor\n",
    "* Divide up responsibilities so that everyone can contribute\n",
    "* Be open when you are struggling to complete a task\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project Timeline Proposal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "| Meeting Date  | Meeting Time| Completed Before Meeting  | Discuss at Meeting |\n",
    "|---|---|---|---|\n",
    "| 1/31 | 11 AM  | N/A  | Previous Project Review | \n",
    "| 2/5  | 10 PM  | Previous Project Review | Potential Project Proposal topics | \n",
    "| 2/8  | 7  PM  | Google doc of potential topics and their relevant datasets | Choosing Project Proposal topic. Assigning roles to complete project proposals by deadline |\n",
    "| 2/13 | 7  PM  | Project Proposal | How to approach web scraping and obtaining the datasets that we need. (tiktok api) Discuss assigning roles for the implementation of the data analysis |\n",
    "| 2/20 | 7  PM  | Each team member's tasks | Checkup on progress, make sure everyone is confident in their tasks |\n",
    "| 2/27 | 7  PM  | Each team member's tasks | Checkup on progress, make sure everyone is confident in their tasks |\n",
    "| 3/5  | 7  PM  | Each team member's tasks  | Checkup on progress, make sure everyone is confident in their tasks |\n",
    "| 3/12 | 7  PM  | The data prediction model is nearing conclusion, time to analyze findings | Outlining the conclusion of the project. Distributing tasks for the final project deliverable with relevant findings |"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}