Cyber_Security_XAI_GAI_web_service

IPS & WAF Detection - attack / normal
WEB Log Detection - SQL Injection, Command Injection, XSS, normal

Data: IPS & WAF Payload, WEB Log
Feature Engineering: PySpark (Spark SQL) based domain ML feature + Automatically generated feature (ex, TF-IDF, ...)
Algorithm: LightGBM, DistilBERT
XAI: ML & DL Shapley value based R&D
Deployment: Flask & Docker
Domain Signature pattern and AI feature highlighting after pattern method matching
WAF & WEB LOG (APACHE or NGINX or IIS) Parsing
WEB LOG based user-agent application, normal_bot_crawler, bad_bot_crawler classification (referenced https://user-agents.net/, https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/_generator_lists/bad-user-agents.list) & Start IP based country name connection (referenced https://dev.maxmind.com/geoip/geolite2-free-geolocation-data)

DistilBERT Transfer Learning (adding cyber security domain word)
DistilBERT (task: question-answering) based fine tuning like SQUAD dataset format
DistilBART based text-summarization
OpenAI API (gpt-3.5-turbo & gpt-4) based XAI analysis
Google API (PALM) test
Cerebras GPT (https://huggingface.co/cerebras/Cerebras-GPT-111M) based Cyber Security domain fine tuning and Gradio based deployment

TO DO: Mitre Att&ck recommandation chatbot FastAPI using OpenAI based LangChain & LangServe & LangSmith

Name		Name	Last commit message	Last commit date
Latest commit History 787 Commits
chat_gpt_context		chat_gpt_context
openai_langchain		openai_langchain
save_model		save_model
sllm_langchain		sllm_langchain
static		static
templates		templates
.DS_Store		.DS_Store
.gitignore		.gitignore
DSS_IPS_Spark_SQL.ipynb		DSS_IPS_Spark_SQL.ipynb
DSS_IPS_shap_explainer_save.py		DSS_IPS_shap_explainer_save.py
DSS_IPS_train.ipynb		DSS_IPS_train.ipynb
DistilBERT_SQUAD_Q&A_fine_tuning.ipynb		DistilBERT_SQUAD_Q&A_fine_tuning.ipynb
DistilBERT_SQUAD_format_convert.ipynb		DistilBERT_SQUAD_format_convert.ipynb
Dockerfile		Dockerfile
LICENSE		LICENSE
Pytorch_DistilBERT_transfer_learning.ipynb		Pytorch_DistilBERT_transfer_learning.ipynb
Pytorch_llm_ko_fine_tuning.ipynb		Pytorch_llm_ko_fine_tuning.ipynb
Pytorch_text_generation_fine_tuning.ipynb		Pytorch_text_generation_fine_tuning.ipynb
README.md		README.md
Transformers_text_summarization.ipynb		Transformers_text_summarization.ipynb
google_generativeai_palm_api.ipynb		google_generativeai_palm_api.ipynb
llm_app.py		llm_app.py
llm_fine_tuning_dataset.ipynb		llm_fine_tuning_dataset.ipynb
requirements.txt		requirements.txt
runserver.py		runserver.py
setting.py		setting.py
shap_explainer_save.py		shap_explainer_save.py