Cloud_Pricing_API

This use case focusses on AWS Price List API & Azure services with JSON file stored on premise, AWS & Azure price listings data was expected as the outcome, extracted the data from AWS API,parsed the complex nested json into a proper schema.Maintained a configuration file to handle the dynamic data, complete distributed computing was ensured.

Key Components:

1. AWS Price List API Integration: We seamlessly integrate with the AWS Price List API to access up-to-date pricing information for AWS services, ensuring accuracy and reliability in our data extraction process.
2. Azure Services Data Retrieval: In addition to AWS, we gather pricing data from Azure services, ensuring comprehensive coverage across major cloud service providers.
3. Data Extraction and Parsing: Our system is equipped to extract data efficiently from both AWS API responses and on-premises JSON files. We employ sophisticated parsing techniques to 4. handle complex nested JSON structures, transforming them into a standardized schema for easy analysis.
4. Dynamic Data Handling: To accommodate dynamic changes in pricing and service offerings, we maintain a configuration file that allows for seamless adjustments and updates to the data extraction process.
5. Distributed Computing: Complete distributed computing architecture is implemented to ensure scalability and performance optimization. By distributing computing tasks across multiple nodes, we achieve efficient processing of large volumes of data.

Expected Outcomes:

1. AWS and Azure Price Listings Data: Comprehensive datasets containing pricing information for AWS and Azure services, extracted and standardized for analysis and comparison. Parsed Data with Standard Schema: Extracted data transformed into a standardized schema, facilitating easy integration with existing analytics pipelines and tools.
2. Dynamic Configuration Handling: Maintenance of a flexible configuration file to adapt to dynamic changes in pricing and service offerings, ensuring data accuracy and relevance over time.
3. Scalable Distributed Computing Architecture: Implementation of a distributed computing framework to handle large-scale data processing, ensuring efficient extraction and parsing of pricing data.

Impact:

By effectively extracting and standardizing pricing information from AWS and Azure services, our project enables organizations to make informed decisions regarding cloud resource allocation and cost optimization. The use of distributed computing ensures scalability and performance, catering to the needs of enterprises with diverse data processing requirements. Ultimately, our efforts contribute to enhancing cost management practices and maximizing ROI in cloud infrastructure utilization.